[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:26:37 dist-test-slave-6bbx krb5kdc[9403](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:26:37 dist-test-slave-6bbx krb5kdc[9403](info): set up 2 sockets
Aug 08 23:26:37 dist-test-slave-6bbx krb5kdc[9403](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 23:26:38.709758 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.320s user 0.007s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:26:38 dist-test-slave-6bbx krb5kdc[9403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:26:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:26:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:26:38.876986 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:39637
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:42525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:39637
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:26:39.421582 9420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:26:39.422055 9420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:26:39.422324 9420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:26:39.431136 9420 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:26:39.431288 9420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:26:39.431402 9420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:26:39.431493 9420 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:26:39.431557 9420 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:26:39.443585 9420 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:42525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:39637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:39637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:26:39.445645 9420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:26:39.447757 9420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:26:39.468708 9429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:39.468622 9432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:39.468698 9434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:26:39.472400 9420 server_base.cc:1034] running on GCE node
I20260808 23:26:39.473549 9420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:26:39.475788 9420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:26:39.477002 9420 hybrid_clock.cc:648] HybridClock initialized: now 1786231599476891 us; error 104 us; skew 500 ppm
I20260808 23:26:39.477583 9420 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:26:39 dist-test-slave-6bbx krb5kdc[9403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231599, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:26:39.481779 9420 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:26:39.484411 9420 webserver.cc:460] Webserver started at http://127.9.44.126:41989/ using document root <none> and password file <none>
I20260808 23:26:39.485474 9420 fs_manager.cc:359] Metadata directory not provided
I20260808 23:26:39.485649 9420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:26:39.486114 9420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:26:39.490355 9420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "8cac7ad7db8f44d3ac6a7b2f6d7b7677"
format_stamp: "Formatted at 2026-08-08 23:26:39 on dist-test-slave-6bbx"
server_key: "4bf3d5742ec52f648cfe491d3ded2a12"
server_key_iv: "378775fafaa88ce8e3e5bda52890c4a7"
server_key_version: "encryptionkey@0"
I20260808 23:26:39.491044 9420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "8cac7ad7db8f44d3ac6a7b2f6d7b7677"
format_stamp: "Formatted at 2026-08-08 23:26:39 on dist-test-slave-6bbx"
server_key: "4bf3d5742ec52f648cfe491d3ded2a12"
server_key_iv: "378775fafaa88ce8e3e5bda52890c4a7"
server_key_version: "encryptionkey@0"
I20260808 23:26:39.499704 9420 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.010s sys 0.000s
I20260808 23:26:39.506094 9442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:39.508059 9420 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.001s
I20260808 23:26:39.508356 9420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "8cac7ad7db8f44d3ac6a7b2f6d7b7677"
format_stamp: "Formatted at 2026-08-08 23:26:39 on dist-test-slave-6bbx"
server_key: "4bf3d5742ec52f648cfe491d3ded2a12"
server_key_iv: "378775fafaa88ce8e3e5bda52890c4a7"
server_key_version: "encryptionkey@0"
I20260808 23:26:39.508563 9420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:26:39.532857 9420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:26:39.538265 9420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:26:39.538682 9420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:26:39.574602 9420 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:39637
I20260808 23:26:39.574720 9521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:39637 every 8 connection(s)
I20260808 23:26:39.577001 9420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:26:39.583451 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 9420
I20260808 23:26:39.583964 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:26:39.585402 9522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:39.587222 9393 external_mini_cluster.cc:1428] Setting key 61d9ff5e04ef054ea6d4633717c70038
I20260808 23:26:39.604622 9522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cac7ad7db8f44d3ac6a7b2f6d7b7677: Bootstrap starting.
I20260808 23:26:39.612685 9522 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cac7ad7db8f44d3ac6a7b2f6d7b7677: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:39.615067 9522 log.cc:824] T 00000000000000000000000000000000 P 8cac7ad7db8f44d3ac6a7b2f6d7b7677: Log is configured to *not* fsync() on all Append() calls
I20260808 23:26:39.620157 9522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cac7ad7db8f44d3ac6a7b2f6d7b7677: No bootstrap required, opened a new log
Aug 08 23:26:39 dist-test-slave-6bbx krb5kdc[9403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:26:39.632527 9522 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cac7ad7db8f44d3ac6a7b2f6d7b7677 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cac7ad7db8f44d3ac6a7b2f6d7b7677" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39637 } }
I20260808 23:26:39.633183 9522 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cac7ad7db8f44d3ac6a7b2f6d7b7677 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:39.633440 9522 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cac7ad7db8f44d3ac6a7b2f6d7b7677 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cac7ad7db8f44d3ac6a7b2f6d7b7677, State: Initialized, Role: FOLLOWER
I20260808 23:26:39.634893 9522 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cac7ad7db8f44d3ac6a7b2f6d7b7677 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cac7ad7db8f44d3ac6a7b2f6d7b7677" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39637 } }
I20260808 23:26:39.635290 9522 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cac7ad7db8f44d3ac6a7b2f6d7b7677 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:26:39.635608 9522 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cac7ad7db8f44d3ac6a7b2f6d7b7677 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:26:39.635907 9522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cac7ad7db8f44d3ac6a7b2f6d7b7677 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:39.638833 9522 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cac7ad7db8f44d3ac6a7b2f6d7b7677 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cac7ad7db8f44d3ac6a7b2f6d7b7677" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39637 } }
I20260808 23:26:39.640136 9522 leader_election.cc:302] T 00000000000000000000000000000000 P 8cac7ad7db8f44d3ac6a7b2f6d7b7677 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cac7ad7db8f44d3ac6a7b2f6d7b7677; no voters:
I20260808 23:26:39.640753 9524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:39.594366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43324 (local address 127.9.44.126:39637)
0808 23:26:39.595808 (+ 1442us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:39.595827 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:39.597661 (+ 1834us) server_negotiation.cc:407] Connection header received
0808 23:26:39.611133 (+ 13472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:39.611162 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:39.611905 (+ 743us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:39.612404 (+ 499us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:39.616677 (+ 4273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:39.618833 (+ 2156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:39.620975 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:39.622315 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:39.622567 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:39.629427 (+ 6860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:39.629496 (+ 69us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:39.629525 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:39.629605 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:39.634493 (+ 4888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:39.635357 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:39.635364 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:39.635370 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:39.635698 (+ 328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:39.636202 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:39.636207 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:39.636210 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:39.637040 (+ 830us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:39.637505 (+ 465us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:39.638549 (+ 1044us) server_negotiation.cc:299] Negotiation successful
0808 23:26:39.638988 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1013,"thread_start_us":622,"threads_started":1}
I20260808 23:26:39.641417 9522 leader_election.cc:288] T 00000000000000000000000000000000 P 8cac7ad7db8f44d3ac6a7b2f6d7b7677 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:26:39.641685 9528 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cac7ad7db8f44d3ac6a7b2f6d7b7677 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:26:39.644235 9528 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cac7ad7db8f44d3ac6a7b2f6d7b7677 [term 1 LEADER]: Becoming Leader. State: Replica: 8cac7ad7db8f44d3ac6a7b2f6d7b7677, State: Running, Role: LEADER
I20260808 23:26:39.645176 9522 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cac7ad7db8f44d3ac6a7b2f6d7b7677 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:26:39.645071 9528 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cac7ad7db8f44d3ac6a7b2f6d7b7677 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cac7ad7db8f44d3ac6a7b2f6d7b7677" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39637 } }
I20260808 23:26:39.652933 9534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cac7ad7db8f44d3ac6a7b2f6d7b7677 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cac7ad7db8f44d3ac6a7b2f6d7b7677. Latest consensus state: current_term: 1 leader_uuid: "8cac7ad7db8f44d3ac6a7b2f6d7b7677" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cac7ad7db8f44d3ac6a7b2f6d7b7677" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39637 } } }
I20260808 23:26:39.653292 9533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cac7ad7db8f44d3ac6a7b2f6d7b7677 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cac7ad7db8f44d3ac6a7b2f6d7b7677" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cac7ad7db8f44d3ac6a7b2f6d7b7677" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39637 } } }
I20260808 23:26:39.653538 9534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cac7ad7db8f44d3ac6a7b2f6d7b7677 [sys.catalog]: This master's current role is: LEADER
I20260808 23:26:39.653626 9533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cac7ad7db8f44d3ac6a7b2f6d7b7677 [sys.catalog]: This master's current role is: LEADER
I20260808 23:26:39.655170 9546 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:26:39.664110 9546 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:26:39.680226 9546 catalog_manager.cc:1347] Generated new cluster ID: 671b4c663d964147ab6f6b02b939161a
I20260808 23:26:39.680361 9546 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:26:39.700147 9546 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:26:39.702194 9546 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:26:39.722755 9546 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cac7ad7db8f44d3ac6a7b2f6d7b7677: Generated new TSK 0
I20260808 23:26:39.724037 9546 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.44.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
I20260808 23:26:39.776903 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:0
--local_ip_for_outbound_sockets=127.9.44.65
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:39637
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:42525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:26:40.314479 9558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:26:40.314935 9558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:26:40.315176 9558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:26:40.324100 9558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:26:40.324291 9558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:26:40.324509 9558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:26:40.336580 9558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:42525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:39637
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:26:40.337713 9558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:26:40.339234 9558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:26:40.356117 9565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:40.356325 9564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:40.356196 9569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:26:40.357774 9558 server_base.cc:1034] running on GCE node
I20260808 23:26:40.358327 9558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:26:40.359292 9558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:26:40.360634 9558 hybrid_clock.cc:648] HybridClock initialized: now 1786231600360495 us; error 43 us; skew 500 ppm
I20260808 23:26:40.361095 9558 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:26:40 dist-test-slave-6bbx krb5kdc[9403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231600, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:26:40.364934 9558 init.cc:377] Logged in from keytab as kudu/127.9.44.65@KRBTEST.COM (short username kudu)
I20260808 23:26:40.366652 9558 webserver.cc:460] Webserver started at http://127.9.44.65:43693/ using document root <none> and password file <none>
I20260808 23:26:40.367276 9558 fs_manager.cc:359] Metadata directory not provided
I20260808 23:26:40.367445 9558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:26:40.367908 9558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:26:40.371430 9558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-0/data/instance:
uuid: "837d618992de490382e216b0c8bbd89e"
format_stamp: "Formatted at 2026-08-08 23:26:40 on dist-test-slave-6bbx"
server_key: "e64151ef37764ce8405d21913d264154"
server_key_iv: "77c043c2ac3f60ed0ac639777e4260ff"
server_key_version: "encryptionkey@0"
I20260808 23:26:40.372236 9558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance:
uuid: "837d618992de490382e216b0c8bbd89e"
format_stamp: "Formatted at 2026-08-08 23:26:40 on dist-test-slave-6bbx"
server_key: "e64151ef37764ce8405d21913d264154"
server_key_iv: "77c043c2ac3f60ed0ac639777e4260ff"
server_key_version: "encryptionkey@0"
I20260808 23:26:40.378248 9558 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260808 23:26:40.382833 9580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:40.384193 9558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 23:26:40.384402 9558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "837d618992de490382e216b0c8bbd89e"
format_stamp: "Formatted at 2026-08-08 23:26:40 on dist-test-slave-6bbx"
server_key: "e64151ef37764ce8405d21913d264154"
server_key_iv: "77c043c2ac3f60ed0ac639777e4260ff"
server_key_version: "encryptionkey@0"
I20260808 23:26:40.384634 9558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:26:40.402841 9558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:26:40.407481 9558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:26:40.407950 9558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:26:40.409737 9558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:26:40.411929 9558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:26:40.412066 9558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:40.412285 9558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:26:40.412388 9558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:40.483196 9558 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:43655
I20260808 23:26:40.483561 9741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:43655 every 8 connection(s)
I20260808 23:26:40.484956 9558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:26:40.487517 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 9558
I20260808 23:26:40.487828 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance
I20260808 23:26:40.488799 9393 external_mini_cluster.cc:1428] Setting key cc6b7bc51d5c66c26a770bbb170c6b7e
Aug 08 23:26:40 dist-test-slave-6bbx krb5kdc[9403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231600, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.44.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.66@KRBTEST.COM" created.
I20260808 23:26:40.513794 9745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:40.489289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:59419 (local address 127.9.44.126:39637)
0808 23:26:40.490076 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:40.490082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:40.491400 (+ 1318us) server_negotiation.cc:407] Connection header received
0808 23:26:40.494488 (+ 3088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:40.494492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:40.494555 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:40.494649 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:40.497058 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:40.498117 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:40.500988 (+ 2871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:40.501582 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:40.501702 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:40.506747 (+ 5045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:40.506753 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:40.506756 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:40.506788 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:40.509941 (+ 3153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:40.510793 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:40.510798 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:40.510800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:40.510978 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:40.511538 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:40.511542 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:40.511544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:40.512008 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:40.512277 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:40.513027 (+ 750us) server_negotiation.cc:299] Negotiation successful
0808 23:26:40.513253 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":473,"threads_started":1}
I20260808 23:26:40.514684 9746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:40.490041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:39637 (local address 127.9.44.65:59419)
0808 23:26:40.491232 (+ 1191us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:40.491286 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:40.494202 (+ 2916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:40.494958 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:40.494970 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:40.495864 (+ 894us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:26:40.496878 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:40.496907 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:40.498296 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:40.498303 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:40.500831 (+ 2528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:40.500847 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:40.501916 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:40.501925 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:40.502411 (+ 486us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:40.503205 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:26:40.503236 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:26:40.506472 (+ 3236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:26:40.510071 (+ 3599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:40.510080 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:40.510094 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:40.510664 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:40.511098 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:40.511103 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:40.511107 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:40.511417 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:40.512135 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:26:40.512142 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:26:40.512524 (+ 382us) client_negotiation.cc:769] Sending connection context
0808 23:26:40.512861 (+ 337us) client_negotiation.cc:240] Negotiation successful
0808 23:26:40.513333 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":859,"thread_start_us":468,"threads_started":1}
I20260808 23:26:40.516650 9742 heartbeater.cc:347] Connected to a master server at 127.9.44.126:39637
I20260808 23:26:40.517105 9742 heartbeater.cc:464] Registering TS with master...
I20260808 23:26:40.518101 9742 heartbeater.cc:511] Master 127.9.44.126:39637 requested a full tablet report, sending...
I20260808 23:26:40.522725 9468 ts_manager.cc:194] Registered new tserver with Master: 837d618992de490382e216b0c8bbd89e (127.9.44.65:43655)
I20260808 23:26:40.526276 9468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.65@KRBTEST.COM'} at 127.9.44.65:59419
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
I20260808 23:26:40.535529 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:0
--local_ip_for_outbound_sockets=127.9.44.66
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:39637
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:42525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:26:41.094025 9751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:26:41.094391 9751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:26:41.094614 9751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:26:41.104477 9751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:26:41.104667 9751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:26:41.104866 9751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:26:41.117146 9751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:42525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:39637
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:26:41.118319 9751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:26:41.119732 9751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:26:41.134979 9760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:41.134948 9761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:41.135638 9763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:26:41.136197 9751 server_base.cc:1034] running on GCE node
I20260808 23:26:41.136924 9751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:26:41.137953 9751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:26:41.139254 9751 hybrid_clock.cc:648] HybridClock initialized: now 1786231601139140 us; error 31 us; skew 500 ppm
I20260808 23:26:41.139608 9751 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:26:41 dist-test-slave-6bbx krb5kdc[9403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231601, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:26:41.143471 9751 init.cc:377] Logged in from keytab as kudu/127.9.44.66@KRBTEST.COM (short username kudu)
I20260808 23:26:41.145534 9751 webserver.cc:460] Webserver started at http://127.9.44.66:41001/ using document root <none> and password file <none>
I20260808 23:26:41.146378 9751 fs_manager.cc:359] Metadata directory not provided
I20260808 23:26:41.146551 9751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:26:41.146945 9751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:26:41.149962 9751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-1/data/instance:
uuid: "1cef20d2162e43e8a58ec3c35e24f2e9"
format_stamp: "Formatted at 2026-08-08 23:26:41 on dist-test-slave-6bbx"
server_key: "ec0122b3d2583f3cb3515aa7b89eaf57"
server_key_iv: "474dd0156183d97a1e77e823d22ff71e"
server_key_version: "encryptionkey@0"
I20260808 23:26:41.150914 9751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance:
uuid: "1cef20d2162e43e8a58ec3c35e24f2e9"
format_stamp: "Formatted at 2026-08-08 23:26:41 on dist-test-slave-6bbx"
server_key: "ec0122b3d2583f3cb3515aa7b89eaf57"
server_key_iv: "474dd0156183d97a1e77e823d22ff71e"
server_key_version: "encryptionkey@0"
I20260808 23:26:41.156636 9751 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 23:26:41.161422 9775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:41.162761 9751 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.001s
I20260808 23:26:41.162958 9751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "1cef20d2162e43e8a58ec3c35e24f2e9"
format_stamp: "Formatted at 2026-08-08 23:26:41 on dist-test-slave-6bbx"
server_key: "ec0122b3d2583f3cb3515aa7b89eaf57"
server_key_iv: "474dd0156183d97a1e77e823d22ff71e"
server_key_version: "encryptionkey@0"
I20260808 23:26:41.163161 9751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:26:41.188272 9751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:26:41.192873 9751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:26:41.193185 9751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:26:41.194309 9751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:26:41.196240 9751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:26:41.196370 9751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:41.196542 9751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:26:41.196607 9751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:41.259680 9751 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:33959
I20260808 23:26:41.259727 9932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:33959 every 8 connection(s)
I20260808 23:26:41.260990 9751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:26:41.263712 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 9751
I20260808 23:26:41.263900 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance
I20260808 23:26:41.264335 9393 external_mini_cluster.cc:1428] Setting key c62b0899f8721516997b708d92b4857d
Aug 08 23:26:41 dist-test-slave-6bbx krb5kdc[9403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231601, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:26:41.286952 9936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:41.264483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:50411 (local address 127.9.44.126:39637)
0808 23:26:41.265098 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:41.265104 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:41.267736 (+ 2632us) server_negotiation.cc:407] Connection header received
0808 23:26:41.269143 (+ 1407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:41.269149 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:41.269232 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:41.269326 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:41.271045 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:41.271943 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:41.273804 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:41.274404 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:41.274521 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:41.279787 (+ 5266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:41.279792 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:41.279795 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:41.279828 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:41.283417 (+ 3589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:41.284505 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:41.284509 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:41.284511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:41.284641 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:41.285216 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:41.285220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:41.285222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:41.285491 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:41.285609 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:41.286439 (+ 830us) server_negotiation.cc:299] Negotiation successful
0808 23:26:41.286597 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":334,"threads_started":1}
WARNING: no policy specified for kudu/127.9.44.67@KRBTEST.COM; defaulting to no policy
I20260808 23:26:41.288244 9937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:41.265155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:39637 (local address 127.9.44.66:50411)
0808 23:26:41.266402 (+ 1247us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:41.266468 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:41.268886 (+ 2418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:41.269572 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:41.269580 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:41.270213 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:26:41.270876 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:41.270895 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:41.272083 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:41.272087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:41.273659 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:41.273677 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:41.274631 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:41.274640 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:41.274993 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:41.275776 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:26:41.275808 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:26:41.279477 (+ 3669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:26:41.283561 (+ 4084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:41.283571 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:41.283586 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:41.284382 (+ 796us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:41.284758 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:41.284763 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:41.284766 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:41.285106 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:41.285624 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:26:41.285632 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:26:41.286028 (+ 396us) client_negotiation.cc:769] Sending connection context
0808 23:26:41.286374 (+ 346us) client_negotiation.cc:240] Negotiation successful
0808 23:26:41.286880 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1007,"thread_start_us":745,"threads_started":1}
I20260808 23:26:41.289700 9933 heartbeater.cc:347] Connected to a master server at 127.9.44.126:39637
I20260808 23:26:41.290097 9933 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.67@KRBTEST.COM" created.
I20260808 23:26:41.291217 9933 heartbeater.cc:511] Master 127.9.44.126:39637 requested a full tablet report, sending...
I20260808 23:26:41.292680 9468 ts_manager.cc:194] Registered new tserver with Master: 1cef20d2162e43e8a58ec3c35e24f2e9 (127.9.44.66:33959)
I20260808 23:26:41.293481 9468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.66@KRBTEST.COM'} at 127.9.44.66:50411
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
I20260808 23:26:41.310446 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:0
--local_ip_for_outbound_sockets=127.9.44.67
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:39637
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:42525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:26:41.530267 9742 heartbeater.cc:503] Master 127.9.44.126:39637 was elected leader, sending a full tablet report...
W20260808 23:26:41.795576 9940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:26:41.795974 9940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:26:41.796125 9940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:26:41.805375 9940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:26:41.805516 9940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:26:41.805683 9940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:26:41.817194 9940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:42525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:39637
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:26:41.818678 9940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:26:41.820446 9940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:26:41.836179 9946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:41.836871 9947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:41.837391 9949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:26:41.843588 9940 server_base.cc:1034] running on GCE node
I20260808 23:26:41.844314 9940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:26:41.845402 9940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:26:41.846649 9940 hybrid_clock.cc:648] HybridClock initialized: now 1786231601846547 us; error 37 us; skew 500 ppm
I20260808 23:26:41.847141 9940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:26:41 dist-test-slave-6bbx krb5kdc[9403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231601, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:26:41.850925 9940 init.cc:377] Logged in from keytab as kudu/127.9.44.67@KRBTEST.COM (short username kudu)
I20260808 23:26:41.852913 9940 webserver.cc:460] Webserver started at http://127.9.44.67:40093/ using document root <none> and password file <none>
I20260808 23:26:41.853681 9940 fs_manager.cc:359] Metadata directory not provided
I20260808 23:26:41.853806 9940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:26:41.854111 9940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:26:41.857306 9940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-2/data/instance:
uuid: "4130bab8e6ed4b58b40b35813e97d45a"
format_stamp: "Formatted at 2026-08-08 23:26:41 on dist-test-slave-6bbx"
server_key: "6cc4cc457d3fc249242374e86f7016aa"
server_key_iv: "007ca8177c825861ce39698dae501b95"
server_key_version: "encryptionkey@0"
I20260808 23:26:41.858285 9940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance:
uuid: "4130bab8e6ed4b58b40b35813e97d45a"
format_stamp: "Formatted at 2026-08-08 23:26:41 on dist-test-slave-6bbx"
server_key: "6cc4cc457d3fc249242374e86f7016aa"
server_key_iv: "007ca8177c825861ce39698dae501b95"
server_key_version: "encryptionkey@0"
I20260808 23:26:41.864408 9940 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260808 23:26:41.868731 9957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:41.870028 9940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:26:41.870190 9940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "4130bab8e6ed4b58b40b35813e97d45a"
format_stamp: "Formatted at 2026-08-08 23:26:41 on dist-test-slave-6bbx"
server_key: "6cc4cc457d3fc249242374e86f7016aa"
server_key_iv: "007ca8177c825861ce39698dae501b95"
server_key_version: "encryptionkey@0"
I20260808 23:26:41.870373 9940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:26:41.882999 9940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:26:41.887583 9940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:26:41.888017 9940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:26:41.889109 9940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:26:41.890590 9940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:26:41.890668 9940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:41.890786 9940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:26:41.890903 9940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:41.954248 9940 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:32987
I20260808 23:26:41.954290 10126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:32987 every 8 connection(s)
I20260808 23:26:41.955714 9940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:26:41.956537 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 9940
I20260808 23:26:41.956854 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance
I20260808 23:26:41.957552 9393 external_mini_cluster.cc:1428] Setting key 46eee66f5715e8630e095ec2455a3c80
Aug 08 23:26:41 dist-test-slave-6bbx krb5kdc[9403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231601, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:26:41.981057 10130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:41.959272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:55077 (local address 127.9.44.126:39637)
0808 23:26:41.959761 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:41.959765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:41.961842 (+ 2077us) server_negotiation.cc:407] Connection header received
0808 23:26:41.964778 (+ 2936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:41.964782 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:41.964847 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:41.964941 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:41.966734 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:41.967903 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:41.969824 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:41.970552 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:41.970670 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:41.974668 (+ 3998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:41.974673 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:41.974676 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:41.974711 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:41.978133 (+ 3422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:41.978835 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:41.978839 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:41.978841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:41.979004 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:41.979412 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:41.979415 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:41.979417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:41.979833 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:41.979942 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:41.980492 (+ 550us) server_negotiation.cc:299] Negotiation successful
0808 23:26:41.980657 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":232,"threads_started":1}
I20260808 23:26:41.982059 10131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:41.960197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:39637 (local address 127.9.44.67:55077)
0808 23:26:41.961676 (+ 1479us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:41.961723 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:41.964512 (+ 2789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:41.965122 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:41.965131 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:41.965775 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:26:41.966560 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:41.966582 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:41.968065 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:41.968069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:41.969657 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:41.969696 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:41.970805 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:41.970811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:41.971107 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:41.971894 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:26:41.971917 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:26:41.974465 (+ 2548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:26:41.978251 (+ 3786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:41.978258 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:41.978269 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:41.978743 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:41.979084 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:41.979087 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:41.979089 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:41.979322 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:41.979939 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:26:41.979944 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:26:41.980215 (+ 271us) client_negotiation.cc:769] Sending connection context
0808 23:26:41.980433 (+ 218us) client_negotiation.cc:240] Negotiation successful
0808 23:26:41.980814 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1139,"thread_start_us":764,"threads_started":1}
I20260808 23:26:41.983314 10128 heartbeater.cc:347] Connected to a master server at 127.9.44.126:39637
I20260808 23:26:41.983585 10128 heartbeater.cc:464] Registering TS with master...
I20260808 23:26:41.984321 10128 heartbeater.cc:511] Master 127.9.44.126:39637 requested a full tablet report, sending...
I20260808 23:26:41.985816 9468 ts_manager.cc:194] Registered new tserver with Master: 4130bab8e6ed4b58b40b35813e97d45a (127.9.44.67:32987)
I20260808 23:26:41.986903 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:26:41.987047 9468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.67@KRBTEST.COM'} at 127.9.44.67:55077
Aug 08 23:26:42 dist-test-slave-6bbx krb5kdc[9403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231602, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 23:26:42 dist-test-slave-6bbx krb5kdc[9403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231602, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.44.65@KRBTEST.COM
I20260808 23:26:42.036098 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:42.018840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34540 (local address 127.9.44.65:43655)
0808 23:26:42.019821 (+ 981us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:42.019828 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:42.019853 (+ 25us) server_negotiation.cc:407] Connection header received
0808 23:26:42.019913 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:42.019919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:42.020112 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:42.020248 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:42.021615 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:42.022659 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:42.024252 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:42.024890 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:42.024988 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:42.029362 (+ 4374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:42.029372 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:42.029385 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:42.029421 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:42.033221 (+ 3800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:42.033795 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:42.033802 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:42.033804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:42.033984 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:42.034278 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:42.034281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:42.034284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:42.034661 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:42.034990 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:42.035335 (+ 345us) server_negotiation.cc:299] Negotiation successful
0808 23:26:42.035560 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":805,"thread_start_us":664,"threads_started":1}
W20260808 23:26:42.037392 9618 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34540
Aug 08 23:26:42 dist-test-slave-6bbx krb5kdc[9403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 23:26:42 dist-test-slave-6bbx krb5kdc[9403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.65@KRBTEST.COM
I20260808 23:26:42.086483 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:42.067509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34542 (local address 127.9.44.65:43655)
0808 23:26:42.067724 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:42.067732 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:42.068033 (+ 301us) server_negotiation.cc:407] Connection header received
0808 23:26:42.068207 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:42.068211 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:42.068276 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:42.068362 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:42.069988 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:42.070687 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:42.072298 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:42.072704 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:42.072816 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:42.079127 (+ 6311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:42.079133 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:42.079138 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:42.079180 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:42.083162 (+ 3982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:42.083855 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:42.083861 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:42.083865 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:42.084125 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:42.084684 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:42.084689 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:42.084693 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:42.085099 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:42.085346 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:42.085826 (+ 480us) server_negotiation.cc:299] Negotiation successful
0808 23:26:42.086132 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 23:26:42.088438 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 9558
I20260808 23:26:42.103965 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:42.089780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34546 (local address 127.9.44.65:43655)
0808 23:26:42.090006 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:42.090013 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:42.090034 (+ 21us) server_negotiation.cc:407] Connection header received
0808 23:26:42.090165 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:42.090170 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:42.090243 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:42.090344 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:42.091572 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:42.092945 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:42.094355 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:42.095114 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:42.095276 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:42.097509 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:42.097516 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:42.097521 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:42.097556 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:42.100887 (+ 3331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:42.101691 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:42.101694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:42.101696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:42.101879 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:42.102426 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:42.102431 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:42.102435 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:42.102751 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:42.102993 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:42.103338 (+ 345us) server_negotiation.cc:299] Negotiation successful
0808 23:26:42.103532 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260808 23:26:42.104705 9736 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:26:42.191879 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 9558
I20260808 23:26:42.214172 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 9751
Aug 08 23:26:42 dist-test-slave-6bbx krb5kdc[9403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.66@KRBTEST.COM
I20260808 23:26:42.229342 10147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:42.214939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58390 (local address 127.9.44.66:33959)
0808 23:26:42.215495 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:42.215502 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:42.215527 (+ 25us) server_negotiation.cc:407] Connection header received
0808 23:26:42.215588 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:42.215595 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:42.215843 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:42.215989 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:42.217199 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:42.218291 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:42.219540 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:42.220231 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:42.220347 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:42.224150 (+ 3803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:42.224161 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:42.224174 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:42.224206 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:42.226663 (+ 2457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:42.227266 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:42.227273 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:42.227275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:42.227410 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:42.227820 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:42.227824 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:42.227826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:42.228152 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:42.228457 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:42.228757 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 23:26:42.229014 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":261,"threads_started":1}
I20260808 23:26:42.230574 9928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:26:42.309043 9933 heartbeater.cc:503] Master 127.9.44.126:39637 was elected leader, sending a full tablet report...
I20260808 23:26:42.310570 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 9751
I20260808 23:26:42.333065 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 9940
Aug 08 23:26:42 dist-test-slave-6bbx krb5kdc[9403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.67@KRBTEST.COM
I20260808 23:26:42.348055 10149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:42.334032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57386 (local address 127.9.44.67:32987)
0808 23:26:42.334606 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:42.334612 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:42.334654 (+ 42us) server_negotiation.cc:407] Connection header received
0808 23:26:42.334711 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:42.334717 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:42.334911 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:42.335030 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:42.336225 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:42.337250 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:42.338443 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:42.338992 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:42.339095 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:42.342957 (+ 3862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:42.342968 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:42.342982 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:42.343014 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:42.345588 (+ 2574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:42.346243 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:42.346250 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:42.346253 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:42.346393 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:42.346690 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:42.346692 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:42.346694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:42.347091 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:42.347326 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:42.347552 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 23:26:42.347778 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":282,"threads_started":1}
I20260808 23:26:42.349268 10121 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:26:42.429853 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 9940
I20260808 23:26:42.449106 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 9420
I20260808 23:26:42.450278 9513 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:26:42.550558 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 9420
2026-08-08T23:26:42Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (5228 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:26:42 dist-test-slave-6bbx krb5kdc[10159](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:26:42 dist-test-slave-6bbx krb5kdc[10159](info): set up 2 sockets
Aug 08 23:26:42 dist-test-slave-6bbx krb5kdc[10159](info): commencing operation
krb5kdc: starting...
W20260808 23:26:43.730142 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:26:43 dist-test-slave-6bbx krb5kdc[10159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:26:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:26:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:26:43.904083 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:38471
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:41715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:38471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:26:44.213380 10182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:26:44.213645 10182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:26:44.213783 10182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:26:44.220712 10182 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:26:44.220813 10182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:26:44.220909 10182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:26:44.220984 10182 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:26:44.221043 10182 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:26:44.230239 10182 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:41715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:38471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:38471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:26:44.231391 10182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:26:44.232597 10182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:26:44.243510 10189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:44.243569 10190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:44.243592 10194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:26:44.244728 10182 server_base.cc:1034] running on GCE node
I20260808 23:26:44.245435 10182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:26:44.246721 10182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:26:44.247917 10182 hybrid_clock.cc:648] HybridClock initialized: now 1786231604247868 us; error 32 us; skew 500 ppm
I20260808 23:26:44.248219 10182 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:26:44 dist-test-slave-6bbx krb5kdc[10159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231604, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:26:44.251139 10182 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:26:44.252532 10182 webserver.cc:460] Webserver started at http://127.9.44.126:38011/ using document root <none> and password file <none>
I20260808 23:26:44.253167 10182 fs_manager.cc:359] Metadata directory not provided
I20260808 23:26:44.253283 10182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:26:44.253564 10182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:26:44.256366 10182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "a2a039dd16634becabdbe38543507ef2"
format_stamp: "Formatted at 2026-08-08 23:26:44 on dist-test-slave-6bbx"
server_key: "088d161a3d79245a9c1ac839111e4b5d"
server_key_iv: "f23b4ed8cb220f3fb978477c2d0d3944"
server_key_version: "encryptionkey@0"
I20260808 23:26:44.257081 10182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "a2a039dd16634becabdbe38543507ef2"
format_stamp: "Formatted at 2026-08-08 23:26:44 on dist-test-slave-6bbx"
server_key: "088d161a3d79245a9c1ac839111e4b5d"
server_key_iv: "f23b4ed8cb220f3fb978477c2d0d3944"
server_key_version: "encryptionkey@0"
I20260808 23:26:44.261755 10182 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 23:26:44.264915 10203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:44.266072 10182 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 23:26:44.266229 10182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "a2a039dd16634becabdbe38543507ef2"
format_stamp: "Formatted at 2026-08-08 23:26:44 on dist-test-slave-6bbx"
server_key: "088d161a3d79245a9c1ac839111e4b5d"
server_key_iv: "f23b4ed8cb220f3fb978477c2d0d3944"
server_key_version: "encryptionkey@0"
I20260808 23:26:44.266399 10182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:26:44.287804 10182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:26:44.291251 10182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:26:44.291495 10182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:26:44.308025 10182 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:38471
I20260808 23:26:44.308077 10273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:38471 every 8 connection(s)
I20260808 23:26:44.309211 10182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:26:44.312618 10274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:44.318213 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 10182
I20260808 23:26:44.318452 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:26:44.319254 9393 external_mini_cluster.cc:1428] Setting key 22a73c3017530e70b630e2133b346177
I20260808 23:26:44.323464 10274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2a039dd16634becabdbe38543507ef2: Bootstrap starting.
I20260808 23:26:44.326923 10274 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2a039dd16634becabdbe38543507ef2: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:44.328025 10274 log.cc:824] T 00000000000000000000000000000000 P a2a039dd16634becabdbe38543507ef2: Log is configured to *not* fsync() on all Append() calls
I20260808 23:26:44.330935 10274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2a039dd16634becabdbe38543507ef2: No bootstrap required, opened a new log
Aug 08 23:26:44 dist-test-slave-6bbx krb5kdc[10159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:26:44.338794 10274 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2a039dd16634becabdbe38543507ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a039dd16634becabdbe38543507ef2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 38471 } }
I20260808 23:26:44.339126 10274 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2a039dd16634becabdbe38543507ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:44.339217 10274 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2a039dd16634becabdbe38543507ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2a039dd16634becabdbe38543507ef2, State: Initialized, Role: FOLLOWER
I20260808 23:26:44.339888 10274 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2a039dd16634becabdbe38543507ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a039dd16634becabdbe38543507ef2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 38471 } }
I20260808 23:26:44.340181 10274 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2a039dd16634becabdbe38543507ef2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:26:44.340322 10274 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2a039dd16634becabdbe38543507ef2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:26:44.340515 10274 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2a039dd16634becabdbe38543507ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:44.341331 10277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:44.321303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34734 (local address 127.9.44.126:38471)
0808 23:26:44.322027 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:44.322037 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:44.322075 (+ 38us) server_negotiation.cc:407] Connection header received
0808 23:26:44.323744 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:44.323767 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:44.324095 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:44.324400 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:44.326162 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:44.327424 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:44.329022 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:44.329715 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:44.329847 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:44.334172 (+ 4325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:44.334179 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:44.334192 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:44.334236 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:44.337183 (+ 2947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:44.337803 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:44.337810 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:44.337819 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:44.337990 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:44.338323 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:44.338326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:44.338328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:44.338791 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:44.339058 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:44.339586 (+ 528us) server_negotiation.cc:299] Negotiation successful
0808 23:26:44.340033 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":331,"threads_started":1}
I20260808 23:26:44.343003 10274 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2a039dd16634becabdbe38543507ef2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a039dd16634becabdbe38543507ef2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 38471 } }
I20260808 23:26:44.343483 10274 leader_election.cc:302] T 00000000000000000000000000000000 P a2a039dd16634becabdbe38543507ef2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2a039dd16634becabdbe38543507ef2; no voters:
I20260808 23:26:44.344101 10274 leader_election.cc:288] T 00000000000000000000000000000000 P a2a039dd16634becabdbe38543507ef2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:26:44.344201 10279 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2a039dd16634becabdbe38543507ef2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:26:44.344817 10279 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2a039dd16634becabdbe38543507ef2 [term 1 LEADER]: Becoming Leader. State: Replica: a2a039dd16634becabdbe38543507ef2, State: Running, Role: LEADER
I20260808 23:26:44.345443 10279 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2a039dd16634becabdbe38543507ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a039dd16634becabdbe38543507ef2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 38471 } }
I20260808 23:26:44.346299 10274 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2a039dd16634becabdbe38543507ef2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:26:44.348086 10281 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2a039dd16634becabdbe38543507ef2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2a039dd16634becabdbe38543507ef2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a039dd16634becabdbe38543507ef2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 38471 } } }
I20260808 23:26:44.348443 10281 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2a039dd16634becabdbe38543507ef2 [sys.catalog]: This master's current role is: LEADER
I20260808 23:26:44.349059 10282 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2a039dd16634becabdbe38543507ef2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2a039dd16634becabdbe38543507ef2. Latest consensus state: current_term: 1 leader_uuid: "a2a039dd16634becabdbe38543507ef2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a039dd16634becabdbe38543507ef2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 38471 } } }
I20260808 23:26:44.349391 10282 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2a039dd16634becabdbe38543507ef2 [sys.catalog]: This master's current role is: LEADER
I20260808 23:26:44.351025 10289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:26:44.358929 10289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:26:44.366708 10289 catalog_manager.cc:1347] Generated new cluster ID: 44e97b4e1f87428bbae10623f4cfd503
I20260808 23:26:44.366815 10289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:26:44.377434 10289 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:26:44.379109 10289 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:26:44.392186 10289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2a039dd16634becabdbe38543507ef2: Generated new TSK 0
I20260808 23:26:44.393071 10289 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.44.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
I20260808 23:26:44.443774 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:0
--local_ip_for_outbound_sockets=127.9.44.65
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:38471
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:41715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:26:44.755256 10305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:26:44.755514 10305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:26:44.755640 10305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:26:44.762477 10305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:26:44.762583 10305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:26:44.762733 10305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:26:44.771916 10305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:41715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:38471
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:26:44.772991 10305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:26:44.774169 10305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:26:44.786170 10312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:44.786214 10316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:44.786240 10314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:26:44.787222 10305 server_base.cc:1034] running on GCE node
I20260808 23:26:44.787797 10305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:26:44.788631 10305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:26:44.789824 10305 hybrid_clock.cc:648] HybridClock initialized: now 1786231604789776 us; error 39 us; skew 500 ppm
I20260808 23:26:44.790096 10305 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:26:44 dist-test-slave-6bbx krb5kdc[10159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231604, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:26:44.793011 10305 init.cc:377] Logged in from keytab as kudu/127.9.44.65@KRBTEST.COM (short username kudu)
I20260808 23:26:44.794382 10305 webserver.cc:460] Webserver started at http://127.9.44.65:33707/ using document root <none> and password file <none>
I20260808 23:26:44.795027 10305 fs_manager.cc:359] Metadata directory not provided
I20260808 23:26:44.795133 10305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:26:44.795382 10305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:26:44.798070 10305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-0/data/instance:
uuid: "5c018651934644e1816ad0958e5d363f"
format_stamp: "Formatted at 2026-08-08 23:26:44 on dist-test-slave-6bbx"
server_key: "a177645dac53000fb1adb92704906ef4"
server_key_iv: "4023ebbaa52e7317b7f062c3ed7f1dc0"
server_key_version: "encryptionkey@0"
I20260808 23:26:44.798770 10305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance:
uuid: "5c018651934644e1816ad0958e5d363f"
format_stamp: "Formatted at 2026-08-08 23:26:44 on dist-test-slave-6bbx"
server_key: "a177645dac53000fb1adb92704906ef4"
server_key_iv: "4023ebbaa52e7317b7f062c3ed7f1dc0"
server_key_version: "encryptionkey@0"
I20260808 23:26:44.803421 10305 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260808 23:26:44.807031 10324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:44.808279 10305 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 23:26:44.808427 10305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "5c018651934644e1816ad0958e5d363f"
format_stamp: "Formatted at 2026-08-08 23:26:44 on dist-test-slave-6bbx"
server_key: "a177645dac53000fb1adb92704906ef4"
server_key_iv: "4023ebbaa52e7317b7f062c3ed7f1dc0"
server_key_version: "encryptionkey@0"
I20260808 23:26:44.808598 10305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:26:44.831319 10305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:26:44.834808 10305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:26:44.835053 10305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:26:44.835939 10305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:26:44.837152 10305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:26:44.837235 10305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:44.837352 10305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:26:44.837404 10305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:44.866823 10305 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:42049
I20260808 23:26:44.866886 10487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:42049 every 8 connection(s)
I20260808 23:26:44.868039 10305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:26:44.868546 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 10305
I20260808 23:26:44.868760 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance
I20260808 23:26:44.869553 9393 external_mini_cluster.cc:1428] Setting key 8b5d4e7786792a259b87930d2eba44de
Aug 08 23:26:44 dist-test-slave-6bbx krb5kdc[10159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231604, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:26:44.887481 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:44.870799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:48251 (local address 127.9.44.126:38471)
0808 23:26:44.871245 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:44.871251 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:44.872374 (+ 1123us) server_negotiation.cc:407] Connection header received
0808 23:26:44.874135 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:44.874141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:44.874226 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:44.874316 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:44.875764 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:44.876481 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:44.877962 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:44.878388 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:44.878498 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:44.882025 (+ 3527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:44.882030 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:44.882033 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:44.882066 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:44.884447 (+ 2381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:44.885209 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:44.885216 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:44.885219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:44.885368 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:44.885754 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:44.885759 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:44.885762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:44.886123 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:44.886285 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:44.886879 (+ 594us) server_negotiation.cc:299] Negotiation successful
0808 23:26:44.887101 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":224,"threads_started":1}
I20260808 23:26:44.888108 10495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:44.871441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:38471 (local address 127.9.44.65:48251)
0808 23:26:44.872228 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:44.872270 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:44.873878 (+ 1608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:44.874550 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:44.874558 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:44.875052 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:26:44.875582 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:44.875600 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:44.876636 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:44.876640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:44.877823 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:44.877836 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:44.878598 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:44.878606 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:44.878823 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:44.879424 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:26:44.879447 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:26:44.881816 (+ 2369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:26:44.884584 (+ 2768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:44.884593 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:44.884608 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:44.885048 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:44.885463 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:44.885467 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:44.885469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:44.885640 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:44.886239 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:26:44.886244 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:26:44.886526 (+ 282us) client_negotiation.cc:769] Sending connection context
0808 23:26:44.886732 (+ 206us) client_negotiation.cc:240] Negotiation successful
0808 23:26:44.887077 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":344,"threads_started":1}
WARNING: no policy specified for kudu/127.9.44.66@KRBTEST.COM; defaulting to no policy
I20260808 23:26:44.889581 10488 heartbeater.cc:347] Connected to a master server at 127.9.44.126:38471
I20260808 23:26:44.889868 10488 heartbeater.cc:464] Registering TS with master...
I20260808 23:26:44.890534 10488 heartbeater.cc:511] Master 127.9.44.126:38471 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.66@KRBTEST.COM" created.
I20260808 23:26:44.892691 10226 ts_manager.cc:194] Registered new tserver with Master: 5c018651934644e1816ad0958e5d363f (127.9.44.65:42049)
I20260808 23:26:44.894091 10226 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.65@KRBTEST.COM'} at 127.9.44.65:48251
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
I20260808 23:26:44.911115 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:0
--local_ip_for_outbound_sockets=127.9.44.66
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:38471
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:41715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:26:45.219743 10498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:26:45.220006 10498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:26:45.220139 10498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:26:45.227073 10498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:26:45.227176 10498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:26:45.227325 10498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:26:45.236445 10498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:41715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:38471
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:26:45.237550 10498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:26:45.238791 10498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:26:45.250874 10505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:45.250952 10509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:45.250917 10512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:26:45.251713 10498 server_base.cc:1034] running on GCE node
I20260808 23:26:45.252352 10498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:26:45.253227 10498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:26:45.254422 10498 hybrid_clock.cc:648] HybridClock initialized: now 1786231605254370 us; error 45 us; skew 500 ppm
I20260808 23:26:45.254700 10498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:26:45 dist-test-slave-6bbx krb5kdc[10159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231605, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:26:45.257838 10498 init.cc:377] Logged in from keytab as kudu/127.9.44.66@KRBTEST.COM (short username kudu)
I20260808 23:26:45.259123 10498 webserver.cc:460] Webserver started at http://127.9.44.66:36109/ using document root <none> and password file <none>
I20260808 23:26:45.259784 10498 fs_manager.cc:359] Metadata directory not provided
I20260808 23:26:45.259897 10498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:26:45.260147 10498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:26:45.262814 10498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-1/data/instance:
uuid: "2415499d9cb44b5bb1ae30bad6c1ae76"
format_stamp: "Formatted at 2026-08-08 23:26:45 on dist-test-slave-6bbx"
server_key: "042fa1eb8a9bd472ce0032f140471f5f"
server_key_iv: "0fa16df87bec88685f4b3ede14446038"
server_key_version: "encryptionkey@0"
I20260808 23:26:45.263552 10498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance:
uuid: "2415499d9cb44b5bb1ae30bad6c1ae76"
format_stamp: "Formatted at 2026-08-08 23:26:45 on dist-test-slave-6bbx"
server_key: "042fa1eb8a9bd472ce0032f140471f5f"
server_key_iv: "0fa16df87bec88685f4b3ede14446038"
server_key_version: "encryptionkey@0"
I20260808 23:26:45.268416 10498 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 23:26:45.271737 10520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:45.272918 10498 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 23:26:45.273059 10498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "2415499d9cb44b5bb1ae30bad6c1ae76"
format_stamp: "Formatted at 2026-08-08 23:26:45 on dist-test-slave-6bbx"
server_key: "042fa1eb8a9bd472ce0032f140471f5f"
server_key_iv: "0fa16df87bec88685f4b3ede14446038"
server_key_version: "encryptionkey@0"
I20260808 23:26:45.273223 10498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:26:45.290302 10498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:26:45.294117 10498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:26:45.294368 10498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:26:45.295280 10498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:26:45.296716 10498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:26:45.296795 10498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:45.296923 10498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:26:45.296976 10498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:45.326200 10498 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:45155
I20260808 23:26:45.326198 10666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:45155 every 8 connection(s)
I20260808 23:26:45.327628 10498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:26:45.335700 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 10498
I20260808 23:26:45.335896 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance
I20260808 23:26:45.336419 9393 external_mini_cluster.cc:1428] Setting key 2e058bc1a0b1fe58e42a18db6a6d3575
Aug 08 23:26:45 dist-test-slave-6bbx krb5kdc[10159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231605, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:26:45.346774 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:45.330036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:36621 (local address 127.9.44.126:38471)
0808 23:26:45.330206 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:45.330212 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:45.331273 (+ 1061us) server_negotiation.cc:407] Connection header received
0808 23:26:45.332978 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:45.332982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:45.333050 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:45.333128 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:45.334436 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:45.335134 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:45.337249 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:45.337697 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:45.337791 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:45.341376 (+ 3585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:45.341381 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:45.341384 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:45.341416 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:45.343913 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:45.344584 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:45.344588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:45.344591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:45.344722 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:45.345132 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:45.345136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:45.345138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:45.345407 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:45.345549 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:45.346209 (+ 660us) server_negotiation.cc:299] Negotiation successful
0808 23:26:45.346440 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 23:26:45.347657 10670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:45.330460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:38471 (local address 127.9.44.66:36621)
0808 23:26:45.331128 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:45.331162 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:45.332737 (+ 1575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:45.333302 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:45.333309 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:45.333795 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:26:45.334312 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:45.334329 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:45.335249 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:45.335252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:45.337109 (+ 1857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:45.337122 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:45.337906 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:45.337913 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:45.338167 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:45.338760 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:26:45.338796 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:26:45.341169 (+ 2373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:26:45.344040 (+ 2871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:45.344049 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:45.344068 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:45.344458 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:45.344825 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:45.344830 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:45.344834 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:45.345016 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:45.345532 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:26:45.345539 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:26:45.345829 (+ 290us) client_negotiation.cc:769] Sending connection context
0808 23:26:45.346078 (+ 249us) client_negotiation.cc:240] Negotiation successful
0808 23:26:45.346562 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":271,"threads_started":1}
I20260808 23:26:45.349021 10667 heartbeater.cc:347] Connected to a master server at 127.9.44.126:38471
I20260808 23:26:45.349321 10667 heartbeater.cc:464] Registering TS with master...
I20260808 23:26:45.350081 10667 heartbeater.cc:511] Master 127.9.44.126:38471 requested a full tablet report, sending...
I20260808 23:26:45.351617 10226 ts_manager.cc:194] Registered new tserver with Master: 2415499d9cb44b5bb1ae30bad6c1ae76 (127.9.44.66:45155)
I20260808 23:26:45.352737 10226 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.66@KRBTEST.COM'} at 127.9.44.66:36621
WARNING: no policy specified for kudu/127.9.44.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
I20260808 23:26:45.376083 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:0
--local_ip_for_outbound_sockets=127.9.44.67
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:38471
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:41715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:26:45.682716 10673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:26:45.682964 10673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:26:45.683060 10673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:26:45.690099 10673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:26:45.690201 10673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:26:45.690323 10673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:26:45.699551 10673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:41715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:38471
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:26:45.700654 10673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:26:45.701874 10673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:26:45.714007 10681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:45.714048 10682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:45.714080 10685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:26:45.714987 10673 server_base.cc:1034] running on GCE node
I20260808 23:26:45.715565 10673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:26:45.716419 10673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:26:45.717617 10673 hybrid_clock.cc:648] HybridClock initialized: now 1786231605717568 us; error 34 us; skew 500 ppm
I20260808 23:26:45.717911 10673 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:26:45 dist-test-slave-6bbx krb5kdc[10159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231605, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:26:45.720825 10673 init.cc:377] Logged in from keytab as kudu/127.9.44.67@KRBTEST.COM (short username kudu)
I20260808 23:26:45.722078 10673 webserver.cc:460] Webserver started at http://127.9.44.67:39617/ using document root <none> and password file <none>
I20260808 23:26:45.722724 10673 fs_manager.cc:359] Metadata directory not provided
I20260808 23:26:45.722841 10673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:26:45.723119 10673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:26:45.725873 10673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-2/data/instance:
uuid: "e9964a794f2641c1a1d9f82c041fe54b"
format_stamp: "Formatted at 2026-08-08 23:26:45 on dist-test-slave-6bbx"
server_key: "d30eb24d9f9505e4a7baecfac62095aa"
server_key_iv: "4c735d7da732583e0a99ecdfef011059"
server_key_version: "encryptionkey@0"
I20260808 23:26:45.726614 10673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance:
uuid: "e9964a794f2641c1a1d9f82c041fe54b"
format_stamp: "Formatted at 2026-08-08 23:26:45 on dist-test-slave-6bbx"
server_key: "d30eb24d9f9505e4a7baecfac62095aa"
server_key_iv: "4c735d7da732583e0a99ecdfef011059"
server_key_version: "encryptionkey@0"
I20260808 23:26:45.731312 10673 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260808 23:26:45.734558 10693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:45.735781 10673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:26:45.735929 10673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "e9964a794f2641c1a1d9f82c041fe54b"
format_stamp: "Formatted at 2026-08-08 23:26:45 on dist-test-slave-6bbx"
server_key: "d30eb24d9f9505e4a7baecfac62095aa"
server_key_iv: "4c735d7da732583e0a99ecdfef011059"
server_key_version: "encryptionkey@0"
I20260808 23:26:45.736096 10673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:26:45.756604 10673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:26:45.760205 10673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:26:45.760458 10673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:26:45.761317 10673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:26:45.762555 10673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:26:45.762635 10673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:45.762754 10673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:26:45.762809 10673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:45.792299 10673 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:45659
I20260808 23:26:45.792374 10850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:45659 every 8 connection(s)
I20260808 23:26:45.793438 10673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:26:45.800045 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 10673
I20260808 23:26:45.800249 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance
I20260808 23:26:45.800789 9393 external_mini_cluster.cc:1428] Setting key f9249867b5bf2fce8d90c6d0ec0abf80
Aug 08 23:26:45 dist-test-slave-6bbx krb5kdc[10159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231605, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:26:45.812170 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:45.795734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:57623 (local address 127.9.44.126:38471)
0808 23:26:45.795892 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:45.795896 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:45.797068 (+ 1172us) server_negotiation.cc:407] Connection header received
0808 23:26:45.798764 (+ 1696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:45.798769 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:45.798837 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:45.798927 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:45.800687 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:45.801468 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:45.803149 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:45.803555 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:45.803693 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:45.806797 (+ 3104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:45.806802 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:45.806805 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:45.806835 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:45.809318 (+ 2483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:45.809947 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:45.809950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:45.809953 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:45.810065 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:45.810430 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:45.810434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:45.810436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:45.810718 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:45.810829 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:45.811549 (+ 720us) server_negotiation.cc:299] Negotiation successful
0808 23:26:45.811836 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 23:26:45.812836 10853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:45.796211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:38471 (local address 127.9.44.67:57623)
0808 23:26:45.796914 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:45.796955 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:45.798528 (+ 1573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:45.799109 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:45.799117 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:45.799761 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:26:45.800518 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:45.800545 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:45.801609 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:45.801613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:45.802999 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:45.803012 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:45.803803 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:45.803810 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:45.804023 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:45.804544 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:26:45.804569 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:26:45.806598 (+ 2029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:26:45.809449 (+ 2851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:45.809456 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:45.809466 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:45.809856 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:45.810137 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:45.810139 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:45.810141 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:45.810309 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:45.810817 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:26:45.810822 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:26:45.811098 (+ 276us) client_negotiation.cc:769] Sending connection context
0808 23:26:45.811338 (+ 240us) client_negotiation.cc:240] Negotiation successful
0808 23:26:45.811826 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":285,"threads_started":1}
I20260808 23:26:45.814071 10851 heartbeater.cc:347] Connected to a master server at 127.9.44.126:38471
I20260808 23:26:45.814368 10851 heartbeater.cc:464] Registering TS with master...
I20260808 23:26:45.815124 10851 heartbeater.cc:511] Master 127.9.44.126:38471 requested a full tablet report, sending...
I20260808 23:26:45.816860 10226 ts_manager.cc:194] Registered new tserver with Master: e9964a794f2641c1a1d9f82c041fe54b (127.9.44.67:45659)
I20260808 23:26:45.817651 10226 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.67@KRBTEST.COM'} at 127.9.44.67:57623
I20260808 23:26:45.827965 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:26:45.850106 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:45.835577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34750 (local address 127.9.44.126:38471)
0808 23:26:45.835763 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:45.835768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:45.836669 (+ 901us) server_negotiation.cc:407] Connection header received
0808 23:26:45.836838 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:45.836842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:45.836902 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:45.836980 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:45.838764 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:45.839693 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:45.840951 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:45.841668 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:45.841764 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:45.844053 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:45.844058 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:45.844061 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:45.844088 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:45.847295 (+ 3207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:45.848017 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:45.848021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:45.848023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:45.848149 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:45.848543 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:45.848549 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:45.848551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:45.848945 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:45.849263 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:45.849606 (+ 343us) server_negotiation.cc:299] Negotiation successful
0808 23:26:45.849817 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 23:26:45.858253 10226 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34750:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:26:45.868057 10226 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:26:45.897015 10488 heartbeater.cc:503] Master 127.9.44.126:38471 was elected leader, sending a full tablet report...
I20260808 23:26:45.914688 10876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:45.885360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59736 (local address 127.9.44.67:45659)
0808 23:26:45.904587 (+ 19227us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:45.904593 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:45.904624 (+ 31us) server_negotiation.cc:407] Connection header received
0808 23:26:45.904735 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:45.904743 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:45.904968 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:45.906156 (+ 1188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:26:45.906423 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:45.908237 (+ 1814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:45.911507 (+ 3270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:45.913195 (+ 1688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:45.913319 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:45.913605 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:45.914220 (+ 615us) server_negotiation.cc:299] Negotiation successful
0808 23:26:45.914376 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19062,"thread_start_us":18875,"threads_started":1}
I20260808 23:26:45.915053 10878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:45.889952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37132 (local address 127.9.44.65:42049)
0808 23:26:45.906354 (+ 16402us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:45.906360 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:45.906380 (+ 20us) server_negotiation.cc:407] Connection header received
0808 23:26:45.906449 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:45.906454 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:45.906592 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:45.906800 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:26:45.907600 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:45.909238 (+ 1638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:45.911610 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:45.913282 (+ 1672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:45.914196 (+ 914us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:45.914414 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:45.914490 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 23:26:45.914650 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16179,"thread_start_us":15915,"threads_started":1}
I20260808 23:26:45.915732 10870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:45.885060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:45659 (local address 127.0.0.1:59736)
0808 23:26:45.898738 (+ 13678us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:45.898778 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:45.899039 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:45.905182 (+ 6143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:45.905193 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:45.905234 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:26:45.905872 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:45.905888 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:45.908369 (+ 2481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:45.908374 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:45.911335 (+ 2961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:45.911350 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:45.913385 (+ 2035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:45.913391 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:45.913593 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:45.913648 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 23:26:45.913812 (+ 164us) client_negotiation.cc:240] Negotiation successful
0808 23:26:45.914028 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13411,"mutex_wait_us":177,"spinlock_wait_cycles":3328,"thread_start_us":14187,"threads_started":1}
I20260808 23:26:45.915933 10877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:45.889589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:42049 (local address 127.0.0.1:37132)
0808 23:26:45.904136 (+ 14547us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:45.904155 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:45.904276 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:45.906766 (+ 2490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:45.906772 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:45.906798 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:26:45.907431 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:45.907443 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:45.909364 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:45.909368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:45.911478 (+ 2110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:45.911494 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:45.913543 (+ 2049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:45.913551 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:45.913737 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:45.913773 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 23:26:45.913841 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 23:26:45.914032 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14422,"mutex_wait_us":71,"thread_start_us":14296,"threads_started":1}
I20260808 23:26:45.916399 10875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:45.885060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:45155 (local address 127.0.0.1:50516)
0808 23:26:45.901719 (+ 16659us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:45.901739 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:45.901860 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:45.905180 (+ 3320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:45.905201 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:45.905243 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:26:45.906080 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:45.906091 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:45.908369 (+ 2278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:45.908374 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:45.911342 (+ 2968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:45.911352 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:45.913961 (+ 2609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:45.913969 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:45.914213 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:45.914243 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 23:26:45.914310 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 23:26:45.914419 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16527,"mutex_wait_us":336,"spinlock_wait_cycles":896,"thread_start_us":16350,"threads_started":1}
I20260808 23:26:45.917120 10872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:45.885039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50516 (local address 127.9.44.66:45155)
0808 23:26:45.904289 (+ 19250us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:45.904298 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:45.904329 (+ 31us) server_negotiation.cc:407] Connection header received
0808 23:26:45.904435 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:45.904443 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:45.904917 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:45.905862 (+ 945us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:26:45.906258 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:45.908206 (+ 1948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:45.911556 (+ 3350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:45.913598 (+ 2042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:45.916307 (+ 2709us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:45.916501 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:45.916578 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 23:26:45.916731 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19063,"thread_start_us":19252,"threads_started":1}
I20260808 23:26:45.920855 10577 tablet_service.cc:1467] Processing CreateTablet for tablet 4147c6331b524cd3b80eeb220fd16931 (DEFAULT_TABLE table=test-table [id=7833dfd594f84a02914831de3f61afda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:26:45.921186 10398 tablet_service.cc:1467] Processing CreateTablet for tablet 4147c6331b524cd3b80eeb220fd16931 (DEFAULT_TABLE table=test-table [id=7833dfd594f84a02914831de3f61afda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:26:45.921474 10760 tablet_service.cc:1467] Processing CreateTablet for tablet 4147c6331b524cd3b80eeb220fd16931 (DEFAULT_TABLE table=test-table [id=7833dfd594f84a02914831de3f61afda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:26:45.922559 10577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4147c6331b524cd3b80eeb220fd16931. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:45.922775 10398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4147c6331b524cd3b80eeb220fd16931. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:45.923213 10760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4147c6331b524cd3b80eeb220fd16931. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:45.936321 10880 tablet_bootstrap.cc:492] T 4147c6331b524cd3b80eeb220fd16931 P 5c018651934644e1816ad0958e5d363f: Bootstrap starting.
I20260808 23:26:45.939062 10881 tablet_bootstrap.cc:492] T 4147c6331b524cd3b80eeb220fd16931 P e9964a794f2641c1a1d9f82c041fe54b: Bootstrap starting.
I20260808 23:26:45.940138 10880 tablet_bootstrap.cc:654] T 4147c6331b524cd3b80eeb220fd16931 P 5c018651934644e1816ad0958e5d363f: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:45.942256 10880 log.cc:824] T 4147c6331b524cd3b80eeb220fd16931 P 5c018651934644e1816ad0958e5d363f: Log is configured to *not* fsync() on all Append() calls
I20260808 23:26:45.943174 10881 tablet_bootstrap.cc:654] T 4147c6331b524cd3b80eeb220fd16931 P e9964a794f2641c1a1d9f82c041fe54b: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:45.944619 10881 log.cc:824] T 4147c6331b524cd3b80eeb220fd16931 P e9964a794f2641c1a1d9f82c041fe54b: Log is configured to *not* fsync() on all Append() calls
I20260808 23:26:45.946501 10880 tablet_bootstrap.cc:492] T 4147c6331b524cd3b80eeb220fd16931 P 5c018651934644e1816ad0958e5d363f: No bootstrap required, opened a new log
I20260808 23:26:45.946874 10880 ts_tablet_manager.cc:1397] T 4147c6331b524cd3b80eeb220fd16931 P 5c018651934644e1816ad0958e5d363f: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.003s
I20260808 23:26:45.948144 10882 tablet_bootstrap.cc:492] T 4147c6331b524cd3b80eeb220fd16931 P 2415499d9cb44b5bb1ae30bad6c1ae76: Bootstrap starting.
I20260808 23:26:45.948223 10881 tablet_bootstrap.cc:492] T 4147c6331b524cd3b80eeb220fd16931 P e9964a794f2641c1a1d9f82c041fe54b: No bootstrap required, opened a new log
I20260808 23:26:45.948589 10881 ts_tablet_manager.cc:1397] T 4147c6331b524cd3b80eeb220fd16931 P e9964a794f2641c1a1d9f82c041fe54b: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260808 23:26:45.950831 10882 tablet_bootstrap.cc:654] T 4147c6331b524cd3b80eeb220fd16931 P 2415499d9cb44b5bb1ae30bad6c1ae76: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:45.952028 10882 log.cc:824] T 4147c6331b524cd3b80eeb220fd16931 P 2415499d9cb44b5bb1ae30bad6c1ae76: Log is configured to *not* fsync() on all Append() calls
I20260808 23:26:45.955195 10882 tablet_bootstrap.cc:492] T 4147c6331b524cd3b80eeb220fd16931 P 2415499d9cb44b5bb1ae30bad6c1ae76: No bootstrap required, opened a new log
I20260808 23:26:45.955436 10882 ts_tablet_manager.cc:1397] T 4147c6331b524cd3b80eeb220fd16931 P 2415499d9cb44b5bb1ae30bad6c1ae76: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 23:26:45.959136 10880 raft_consensus.cc:348] T 4147c6331b524cd3b80eeb220fd16931 P 5c018651934644e1816ad0958e5d363f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2415499d9cb44b5bb1ae30bad6c1ae76" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45155 } } peers { permanent_uuid: "e9964a794f2641c1a1d9f82c041fe54b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 45659 } } peers { permanent_uuid: "5c018651934644e1816ad0958e5d363f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42049 } }
I20260808 23:26:45.959739 10880 raft_consensus.cc:374] T 4147c6331b524cd3b80eeb220fd16931 P 5c018651934644e1816ad0958e5d363f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:45.959852 10880 raft_consensus.cc:729] T 4147c6331b524cd3b80eeb220fd16931 P 5c018651934644e1816ad0958e5d363f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c018651934644e1816ad0958e5d363f, State: Initialized, Role: FOLLOWER
I20260808 23:26:45.960737 10880 consensus_queue.cc:260] T 4147c6331b524cd3b80eeb220fd16931 P 5c018651934644e1816ad0958e5d363f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2415499d9cb44b5bb1ae30bad6c1ae76" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45155 } } peers { permanent_uuid: "e9964a794f2641c1a1d9f82c041fe54b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 45659 } } peers { permanent_uuid: "5c018651934644e1816ad0958e5d363f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42049 } }
I20260808 23:26:45.960878 10881 raft_consensus.cc:348] T 4147c6331b524cd3b80eeb220fd16931 P e9964a794f2641c1a1d9f82c041fe54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2415499d9cb44b5bb1ae30bad6c1ae76" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45155 } } peers { permanent_uuid: "e9964a794f2641c1a1d9f82c041fe54b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 45659 } } peers { permanent_uuid: "5c018651934644e1816ad0958e5d363f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42049 } }
I20260808 23:26:45.961310 10881 raft_consensus.cc:374] T 4147c6331b524cd3b80eeb220fd16931 P e9964a794f2641c1a1d9f82c041fe54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:45.961428 10881 raft_consensus.cc:729] T 4147c6331b524cd3b80eeb220fd16931 P e9964a794f2641c1a1d9f82c041fe54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9964a794f2641c1a1d9f82c041fe54b, State: Initialized, Role: FOLLOWER
I20260808 23:26:45.962355 10881 consensus_queue.cc:260] T 4147c6331b524cd3b80eeb220fd16931 P e9964a794f2641c1a1d9f82c041fe54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2415499d9cb44b5bb1ae30bad6c1ae76" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45155 } } peers { permanent_uuid: "e9964a794f2641c1a1d9f82c041fe54b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 45659 } } peers { permanent_uuid: "5c018651934644e1816ad0958e5d363f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42049 } }
I20260808 23:26:45.963187 10880 ts_tablet_manager.cc:1428] T 4147c6331b524cd3b80eeb220fd16931 P 5c018651934644e1816ad0958e5d363f: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
I20260808 23:26:45.964350 10851 heartbeater.cc:503] Master 127.9.44.126:38471 was elected leader, sending a full tablet report...
I20260808 23:26:45.965852 10881 ts_tablet_manager.cc:1428] T 4147c6331b524cd3b80eeb220fd16931 P e9964a794f2641c1a1d9f82c041fe54b: Time spent starting tablet: real 0.017s user 0.014s sys 0.002s
I20260808 23:26:45.968222 10882 raft_consensus.cc:348] T 4147c6331b524cd3b80eeb220fd16931 P 2415499d9cb44b5bb1ae30bad6c1ae76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2415499d9cb44b5bb1ae30bad6c1ae76" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45155 } } peers { permanent_uuid: "e9964a794f2641c1a1d9f82c041fe54b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 45659 } } peers { permanent_uuid: "5c018651934644e1816ad0958e5d363f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42049 } }
I20260808 23:26:45.968907 10882 raft_consensus.cc:374] T 4147c6331b524cd3b80eeb220fd16931 P 2415499d9cb44b5bb1ae30bad6c1ae76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:45.969102 10882 raft_consensus.cc:729] T 4147c6331b524cd3b80eeb220fd16931 P 2415499d9cb44b5bb1ae30bad6c1ae76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2415499d9cb44b5bb1ae30bad6c1ae76, State: Initialized, Role: FOLLOWER
I20260808 23:26:45.969729 10882 consensus_queue.cc:260] T 4147c6331b524cd3b80eeb220fd16931 P 2415499d9cb44b5bb1ae30bad6c1ae76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2415499d9cb44b5bb1ae30bad6c1ae76" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45155 } } peers { permanent_uuid: "e9964a794f2641c1a1d9f82c041fe54b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 45659 } } peers { permanent_uuid: "5c018651934644e1816ad0958e5d363f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42049 } }
I20260808 23:26:45.971274 10667 heartbeater.cc:503] Master 127.9.44.126:38471 was elected leader, sending a full tablet report...
I20260808 23:26:45.971809 10882 ts_tablet_manager.cc:1428] T 4147c6331b524cd3b80eeb220fd16931 P 2415499d9cb44b5bb1ae30bad6c1ae76: Time spent starting tablet: real 0.016s user 0.010s sys 0.007s
W20260808 23:26:46.044941 10852 tablet.cc:2369] T 4147c6331b524cd3b80eeb220fd16931 P e9964a794f2641c1a1d9f82c041fe54b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:26:46.079613 10669 tablet.cc:2369] T 4147c6331b524cd3b80eeb220fd16931 P 2415499d9cb44b5bb1ae30bad6c1ae76: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:26:46.120445 10489 tablet.cc:2369] T 4147c6331b524cd3b80eeb220fd16931 P 5c018651934644e1816ad0958e5d363f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:26:46.279002 10888 raft_consensus.cc:482] T 4147c6331b524cd3b80eeb220fd16931 P e9964a794f2641c1a1d9f82c041fe54b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:26:46.279312 10888 raft_consensus.cc:504] T 4147c6331b524cd3b80eeb220fd16931 P e9964a794f2641c1a1d9f82c041fe54b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2415499d9cb44b5bb1ae30bad6c1ae76" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45155 } } peers { permanent_uuid: "e9964a794f2641c1a1d9f82c041fe54b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 45659 } } peers { permanent_uuid: "5c018651934644e1816ad0958e5d363f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42049 } }
I20260808 23:26:46.281783 10888 leader_election.cc:288] T 4147c6331b524cd3b80eeb220fd16931 P e9964a794f2641c1a1d9f82c041fe54b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2415499d9cb44b5bb1ae30bad6c1ae76 (127.9.44.66:45155), 5c018651934644e1816ad0958e5d363f (127.9.44.65:42049)
I20260808 23:26:46.288729 10872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:46.282061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:41943 (local address 127.9.44.66:45155)
0808 23:26:46.282294 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:46.282300 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:46.282404 (+ 104us) server_negotiation.cc:407] Connection header received
0808 23:26:46.282612 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:46.282618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:46.282694 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:46.282794 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:26:46.283374 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:46.284587 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:46.286482 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:46.287859 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:46.287999 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:46.288089 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:46.288442 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 23:26:46.288528 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 23:26:46.288898 10853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:46.282148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:45155 (local address 127.9.44.67:41943)
0808 23:26:46.282315 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:46.282337 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:46.282467 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:46.282932 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:46.282937 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:46.282971 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:26:46.283219 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:46.283235 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:46.284738 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:46.284744 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:46.286321 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:46.286334 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:46.288089 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:46.288095 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:46.288277 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:46.288311 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 23:26:46.288428 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 23:26:46.288561 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":6}
I20260808 23:26:46.289918 10890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:46.282356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:42049 (local address 127.9.44.67:50323)
0808 23:26:46.283146 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:46.283159 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:46.283283 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:46.283601 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:46.283605 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:46.283631 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:26:46.284143 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:46.284156 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:46.285497 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:46.285501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:46.287470 (+ 1969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:46.287486 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:46.289182 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:46.289188 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:46.289323 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:46.289346 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 23:26:46.289433 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 23:26:46.289552 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":411,"threads_started":1}
I20260808 23:26:46.290272 10606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4147c6331b524cd3b80eeb220fd16931" candidate_uuid: "e9964a794f2641c1a1d9f82c041fe54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2415499d9cb44b5bb1ae30bad6c1ae76" is_pre_election: true
I20260808 23:26:46.290956 10878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:46.282617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:50323 (local address 127.9.44.65:42049)
0808 23:26:46.282832 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:46.282838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:46.283249 (+ 411us) server_negotiation.cc:407] Connection header received
0808 23:26:46.283430 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:46.283434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:46.283494 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:46.283561 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:26:46.284310 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:46.285346 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:46.287630 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:46.288980 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:46.289117 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:46.289213 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:46.290684 (+ 1471us) server_negotiation.cc:299] Negotiation successful
0808 23:26:46.290788 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 23:26:46.291100 10606 raft_consensus.cc:2454] T 4147c6331b524cd3b80eeb220fd16931 P 2415499d9cb44b5bb1ae30bad6c1ae76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9964a794f2641c1a1d9f82c041fe54b in term 0.
I20260808 23:26:46.291950 10695 leader_election.cc:302] T 4147c6331b524cd3b80eeb220fd16931 P e9964a794f2641c1a1d9f82c041fe54b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2415499d9cb44b5bb1ae30bad6c1ae76, e9964a794f2641c1a1d9f82c041fe54b; no voters:
I20260808 23:26:46.291843 10426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4147c6331b524cd3b80eeb220fd16931" candidate_uuid: "e9964a794f2641c1a1d9f82c041fe54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c018651934644e1816ad0958e5d363f" is_pre_election: true
I20260808 23:26:46.292347 10888 raft_consensus.cc:2781] T 4147c6331b524cd3b80eeb220fd16931 P e9964a794f2641c1a1d9f82c041fe54b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:26:46.292542 10888 raft_consensus.cc:482] T 4147c6331b524cd3b80eeb220fd16931 P e9964a794f2641c1a1d9f82c041fe54b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:26:46.292660 10888 raft_consensus.cc:3037] T 4147c6331b524cd3b80eeb220fd16931 P e9964a794f2641c1a1d9f82c041fe54b [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:46.292670 10426 raft_consensus.cc:2454] T 4147c6331b524cd3b80eeb220fd16931 P 5c018651934644e1816ad0958e5d363f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9964a794f2641c1a1d9f82c041fe54b in term 0.
I20260808 23:26:46.295883 10888 raft_consensus.cc:504] T 4147c6331b524cd3b80eeb220fd16931 P e9964a794f2641c1a1d9f82c041fe54b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2415499d9cb44b5bb1ae30bad6c1ae76" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45155 } } peers { permanent_uuid: "e9964a794f2641c1a1d9f82c041fe54b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 45659 } } peers { permanent_uuid: "5c018651934644e1816ad0958e5d363f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42049 } }
I20260808 23:26:46.296738 10888 leader_election.cc:288] T 4147c6331b524cd3b80eeb220fd16931 P e9964a794f2641c1a1d9f82c041fe54b [CANDIDATE]: Term 1 election: Requested vote from peers 2415499d9cb44b5bb1ae30bad6c1ae76 (127.9.44.66:45155), 5c018651934644e1816ad0958e5d363f (127.9.44.65:42049)
I20260808 23:26:46.297155 10606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4147c6331b524cd3b80eeb220fd16931" candidate_uuid: "e9964a794f2641c1a1d9f82c041fe54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2415499d9cb44b5bb1ae30bad6c1ae76"
I20260808 23:26:46.297168 10426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4147c6331b524cd3b80eeb220fd16931" candidate_uuid: "e9964a794f2641c1a1d9f82c041fe54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c018651934644e1816ad0958e5d363f"
I20260808 23:26:46.297364 10426 raft_consensus.cc:3037] T 4147c6331b524cd3b80eeb220fd16931 P 5c018651934644e1816ad0958e5d363f [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:46.297437 10606 raft_consensus.cc:3037] T 4147c6331b524cd3b80eeb220fd16931 P 2415499d9cb44b5bb1ae30bad6c1ae76 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:46.300581 10426 raft_consensus.cc:2454] T 4147c6331b524cd3b80eeb220fd16931 P 5c018651934644e1816ad0958e5d363f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9964a794f2641c1a1d9f82c041fe54b in term 1.
I20260808 23:26:46.300849 10606 raft_consensus.cc:2454] T 4147c6331b524cd3b80eeb220fd16931 P 2415499d9cb44b5bb1ae30bad6c1ae76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9964a794f2641c1a1d9f82c041fe54b in term 1.
I20260808 23:26:46.300995 10694 leader_election.cc:302] T 4147c6331b524cd3b80eeb220fd16931 P e9964a794f2641c1a1d9f82c041fe54b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c018651934644e1816ad0958e5d363f, e9964a794f2641c1a1d9f82c041fe54b; no voters:
I20260808 23:26:46.301390 10888 raft_consensus.cc:2781] T 4147c6331b524cd3b80eeb220fd16931 P e9964a794f2641c1a1d9f82c041fe54b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:26:46.301785 10888 raft_consensus.cc:686] T 4147c6331b524cd3b80eeb220fd16931 P e9964a794f2641c1a1d9f82c041fe54b [term 1 LEADER]: Becoming Leader. State: Replica: e9964a794f2641c1a1d9f82c041fe54b, State: Running, Role: LEADER
I20260808 23:26:46.302239 10888 consensus_queue.cc:237] T 4147c6331b524cd3b80eeb220fd16931 P e9964a794f2641c1a1d9f82c041fe54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2415499d9cb44b5bb1ae30bad6c1ae76" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45155 } } peers { permanent_uuid: "e9964a794f2641c1a1d9f82c041fe54b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 45659 } } peers { permanent_uuid: "5c018651934644e1816ad0958e5d363f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42049 } }
I20260808 23:26:46.307775 10224 catalog_manager.cc:5495] T 4147c6331b524cd3b80eeb220fd16931 P e9964a794f2641c1a1d9f82c041fe54b reported cstate change: term changed from 0 to 1, leader changed from <none> to e9964a794f2641c1a1d9f82c041fe54b (127.9.44.67). New cstate: current_term: 1 leader_uuid: "e9964a794f2641c1a1d9f82c041fe54b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2415499d9cb44b5bb1ae30bad6c1ae76" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9964a794f2641c1a1d9f82c041fe54b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 45659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c018651934644e1816ad0958e5d363f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42049 } health_report { overall_health: UNKNOWN } } }
Aug 08 23:26:46 dist-test-slave-6bbx krb5kdc[10159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.65@KRBTEST.COM
I20260808 23:26:46.352510 10878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:46.333804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37146 (local address 127.9.44.65:42049)
0808 23:26:46.333951 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:46.333955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:46.335329 (+ 1374us) server_negotiation.cc:407] Connection header received
0808 23:26:46.335483 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:46.335487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:46.335557 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:46.335641 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:46.337447 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:46.338439 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:46.340411 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:46.340844 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:46.340957 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:46.346305 (+ 5348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:46.346319 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:46.346333 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:46.346365 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:46.349574 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:46.350257 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:46.350264 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:46.350266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:46.350445 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:46.350868 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:46.350872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:46.350874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:46.351325 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:46.351606 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:46.351962 (+ 356us) server_negotiation.cc:299] Negotiation successful
0808 23:26:46.352183 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 23:26:46 dist-test-slave-6bbx krb5kdc[10159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231606, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 23:26:46 dist-test-slave-6bbx krb5kdc[10159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231606, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.44.65@KRBTEST.COM
I20260808 23:26:46.402436 10878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:46.386800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37158 (local address 127.9.44.65:42049)
0808 23:26:46.386984 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:46.386991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:46.387441 (+ 450us) server_negotiation.cc:407] Connection header received
0808 23:26:46.387629 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:46.387633 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:46.387726 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:46.387814 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:46.389454 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:46.390231 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:46.391438 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:46.391993 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:46.392107 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:46.396356 (+ 4249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:46.396361 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:46.396364 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:46.396395 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:46.399757 (+ 3362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:46.400432 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:46.400438 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:46.400442 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:46.400582 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:46.400941 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:46.400946 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:46.400950 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:46.401211 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:46.401550 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:46.401758 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 23:26:46.402014 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 23:26:46 dist-test-slave-6bbx krb5kdc[10159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 23:26:46 dist-test-slave-6bbx krb5kdc[10159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.65@KRBTEST.COM
I20260808 23:26:46.448244 10878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:46.433667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37160 (local address 127.9.44.65:42049)
0808 23:26:46.433910 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:46.433917 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:46.434141 (+ 224us) server_negotiation.cc:407] Connection header received
0808 23:26:46.434303 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:46.434307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:46.434370 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:46.434455 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:46.435918 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:46.436854 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:46.438418 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:46.438989 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:46.439103 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:46.443149 (+ 4046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:46.443154 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:46.443157 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:46.443196 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:46.445813 (+ 2617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:46.446399 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:46.446402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:46.446404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:46.446537 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:46.446882 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:46.446886 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:46.446888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:46.447191 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:46.447558 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:46.447708 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 23:26:46.447933 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 23:26:46.450270 10365 tablet_service.cc:2908] Rejecting scan request for tablet 4147c6331b524cd3b80eeb220fd16931: Uninitialized: clean time has not yet been initialized
I20260808 23:26:46.453715 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 10305
I20260808 23:26:46.465796 10878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:46.454569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37170 (local address 127.9.44.65:42049)
0808 23:26:46.454702 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:46.454705 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:46.454996 (+ 291us) server_negotiation.cc:407] Connection header received
0808 23:26:46.455109 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:46.455112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:46.455176 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:46.455247 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:46.456384 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:46.457026 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:46.458087 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:46.458461 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:46.458557 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:46.461131 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:46.461135 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:46.461139 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:46.461164 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:46.463508 (+ 2344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:46.464138 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:46.464142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:46.464144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:46.464250 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:46.464607 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:46.464611 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:46.464614 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:46.464945 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:46.465139 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:46.465372 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 23:26:46.465526 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 23:26:46.466346 10481 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:26:46.553439 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 10305
I20260808 23:26:46.576575 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 10498
Aug 08 23:26:46 dist-test-slave-6bbx krb5kdc[10159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.66@KRBTEST.COM
I20260808 23:26:46.591399 10872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:46.577343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50526 (local address 127.9.44.66:45155)
0808 23:26:46.577526 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:46.577532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:46.577568 (+ 36us) server_negotiation.cc:407] Connection header received
0808 23:26:46.577756 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:46.577762 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:46.577839 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:46.577943 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:46.579140 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:46.579974 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:46.581234 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:46.581625 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:46.581741 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:46.585579 (+ 3838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:46.585594 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:46.585611 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:46.585644 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:46.588551 (+ 2907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:46.589181 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:46.589189 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:46.589192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:46.589363 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:46.589722 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:46.589727 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:46.589729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:46.590286 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:46.590592 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:46.590878 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 23:26:46.591084 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 23:26:46.592125 10658 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:26:46.676465 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 10498
I20260808 23:26:46.698022 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 10673
Aug 08 23:26:46 dist-test-slave-6bbx krb5kdc[10159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.67@KRBTEST.COM
I20260808 23:26:46.711594 10917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:46.698745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59744 (local address 127.9.44.67:45659)
0808 23:26:46.699212 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:46.699217 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:46.699233 (+ 16us) server_negotiation.cc:407] Connection header received
0808 23:26:46.699290 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:46.699293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:46.699359 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:46.699447 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:46.700662 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:46.701356 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:46.702517 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:46.702902 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:46.702991 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:46.706822 (+ 3831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:46.706836 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:46.706854 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:46.706901 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:46.709424 (+ 2523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:46.709984 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:46.709991 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:46.709993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:46.710127 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:46.710433 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:46.710438 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:46.710441 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:46.710783 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:46.710997 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:46.711160 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 23:26:46.711313 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":254,"threads_started":1}
I20260808 23:26:46.712335 10844 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:26:46.798801 10920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:46.797975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:42049 (local address 127.9.44.67:58633)
0808 23:26:46.798471 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:46.798620 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:42049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":273,"threads_started":1}
W20260808 23:26:46.799058 10694 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.44.65:42049: connect: Connection refused (error 111)
I20260808 23:26:46.808485 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 10673
I20260808 23:26:46.831135 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 10182
I20260808 23:26:46.832279 10267 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:26:46.939764 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 10182
2026-08-08T23:26:46Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4391 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:26:47 dist-test-slave-6bbx krb5kdc[10927](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:26:47 dist-test-slave-6bbx krb5kdc[10927](info): set up 2 sockets
Aug 08 23:26:47 dist-test-slave-6bbx krb5kdc[10927](info): commencing operation
krb5kdc: starting...
W20260808 23:26:48.117115 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:26:48 dist-test-slave-6bbx krb5kdc[10927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:26:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:26:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:26:48.284937 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:40739
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:45445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:40739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:26:48.592386 10943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:26:48.592641 10943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:26:48.592767 10943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:26:48.599573 10943 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:26:48.599682 10943 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 23:26:48.599753 10943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:26:48.599829 10943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:26:48.599895 10943 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:26:48.599977 10943 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:26:48.609202 10943 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:45445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:40739
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:40739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:26:48.610357 10943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:26:48.611631 10943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:26:48.622718 10949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:48.622767 10952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:48.622692 10948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:26:48.623536 10943 server_base.cc:1034] running on GCE node
I20260808 23:26:48.624269 10943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:26:48.625522 10943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:26:48.626739 10943 hybrid_clock.cc:648] HybridClock initialized: now 1786231608626674 us; error 26 us; skew 500 ppm
I20260808 23:26:48.627045 10943 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:26:48 dist-test-slave-6bbx krb5kdc[10927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231608, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:26:48.632905 10943 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:26:48.634184 10943 webserver.cc:460] Webserver started at http://127.9.44.126:40587/ using document root <none> and password file <none>
I20260808 23:26:48.634840 10943 fs_manager.cc:359] Metadata directory not provided
I20260808 23:26:48.634933 10943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:26:48.635239 10943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:26:48.638043 10943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "723cab76a5c14d61bb8fffc84a3ea699"
format_stamp: "Formatted at 2026-08-08 23:26:48 on dist-test-slave-6bbx"
server_key: "92b8237b3576d8b4bc596e700c22f65e"
server_key_iv: "92851b64313f4c60f9ad3b94de40d958"
server_key_version: "encryptionkey@0"
I20260808 23:26:48.638798 10943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "723cab76a5c14d61bb8fffc84a3ea699"
format_stamp: "Formatted at 2026-08-08 23:26:48 on dist-test-slave-6bbx"
server_key: "92b8237b3576d8b4bc596e700c22f65e"
server_key_iv: "92851b64313f4c60f9ad3b94de40d958"
server_key_version: "encryptionkey@0"
I20260808 23:26:48.643486 10943 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 23:26:48.646654 10961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:48.647890 10943 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:26:48.648042 10943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "723cab76a5c14d61bb8fffc84a3ea699"
format_stamp: "Formatted at 2026-08-08 23:26:48 on dist-test-slave-6bbx"
server_key: "92b8237b3576d8b4bc596e700c22f65e"
server_key_iv: "92851b64313f4c60f9ad3b94de40d958"
server_key_version: "encryptionkey@0"
I20260808 23:26:48.648222 10943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:26:48.664142 10943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:26:48.667498 10943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:26:48.667757 10943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:26:48.687603 10943 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:40739
I20260808 23:26:48.687642 11049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:40739 every 8 connection(s)
I20260808 23:26:48.688869 10943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:26:48.692320 11050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:48.697870 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 10943
I20260808 23:26:48.698094 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:26:48.698889 9393 external_mini_cluster.cc:1428] Setting key b89209511f5cf29e9673445a2608dc74
I20260808 23:26:48.703111 11050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 723cab76a5c14d61bb8fffc84a3ea699: Bootstrap starting.
I20260808 23:26:48.706606 11050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 723cab76a5c14d61bb8fffc84a3ea699: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:48.707777 11050 log.cc:824] T 00000000000000000000000000000000 P 723cab76a5c14d61bb8fffc84a3ea699: Log is configured to *not* fsync() on all Append() calls
I20260808 23:26:48.710662 11050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 723cab76a5c14d61bb8fffc84a3ea699: No bootstrap required, opened a new log
Aug 08 23:26:48 dist-test-slave-6bbx krb5kdc[10927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:26:48.719105 11050 raft_consensus.cc:348] T 00000000000000000000000000000000 P 723cab76a5c14d61bb8fffc84a3ea699 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "723cab76a5c14d61bb8fffc84a3ea699" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 40739 } }
I20260808 23:26:48.719539 11050 raft_consensus.cc:374] T 00000000000000000000000000000000 P 723cab76a5c14d61bb8fffc84a3ea699 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:48.719651 11050 raft_consensus.cc:729] T 00000000000000000000000000000000 P 723cab76a5c14d61bb8fffc84a3ea699 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 723cab76a5c14d61bb8fffc84a3ea699, State: Initialized, Role: FOLLOWER
I20260808 23:26:48.720273 11050 consensus_queue.cc:260] T 00000000000000000000000000000000 P 723cab76a5c14d61bb8fffc84a3ea699 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "723cab76a5c14d61bb8fffc84a3ea699" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 40739 } }
I20260808 23:26:48.720513 11050 raft_consensus.cc:388] T 00000000000000000000000000000000 P 723cab76a5c14d61bb8fffc84a3ea699 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:26:48.720649 11050 raft_consensus.cc:482] T 00000000000000000000000000000000 P 723cab76a5c14d61bb8fffc84a3ea699 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:26:48.720816 11050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 723cab76a5c14d61bb8fffc84a3ea699 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:48.720983 11055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:48.700890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38548 (local address 127.9.44.126:40739)
0808 23:26:48.701629 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:48.701640 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:48.701679 (+ 39us) server_negotiation.cc:407] Connection header received
0808 23:26:48.703293 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:48.703317 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:48.703653 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:48.703981 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:48.705725 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:48.707006 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:48.708670 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:48.709406 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:48.709530 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:48.714187 (+ 4657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:48.714194 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:48.714216 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:48.714251 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:48.717134 (+ 2883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:48.717892 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:48.717898 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:48.717903 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:48.718062 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:48.718426 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:48.718429 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:48.718431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:48.718904 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:48.719114 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:48.719546 (+ 432us) server_negotiation.cc:299] Negotiation successful
0808 23:26:48.719930 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":353,"threads_started":1}
I20260808 23:26:48.722998 11050 raft_consensus.cc:504] T 00000000000000000000000000000000 P 723cab76a5c14d61bb8fffc84a3ea699 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "723cab76a5c14d61bb8fffc84a3ea699" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 40739 } }
I20260808 23:26:48.723526 11050 leader_election.cc:302] T 00000000000000000000000000000000 P 723cab76a5c14d61bb8fffc84a3ea699 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 723cab76a5c14d61bb8fffc84a3ea699; no voters:
I20260808 23:26:48.724066 11050 leader_election.cc:288] T 00000000000000000000000000000000 P 723cab76a5c14d61bb8fffc84a3ea699 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:26:48.724169 11061 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 723cab76a5c14d61bb8fffc84a3ea699 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:26:48.724782 11061 raft_consensus.cc:686] T 00000000000000000000000000000000 P 723cab76a5c14d61bb8fffc84a3ea699 [term 1 LEADER]: Becoming Leader. State: Replica: 723cab76a5c14d61bb8fffc84a3ea699, State: Running, Role: LEADER
I20260808 23:26:48.725291 11061 consensus_queue.cc:237] T 00000000000000000000000000000000 P 723cab76a5c14d61bb8fffc84a3ea699 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "723cab76a5c14d61bb8fffc84a3ea699" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 40739 } }
I20260808 23:26:48.725907 11050 sys_catalog.cc:564] T 00000000000000000000000000000000 P 723cab76a5c14d61bb8fffc84a3ea699 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:26:48.727615 11064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 723cab76a5c14d61bb8fffc84a3ea699 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 723cab76a5c14d61bb8fffc84a3ea699. Latest consensus state: current_term: 1 leader_uuid: "723cab76a5c14d61bb8fffc84a3ea699" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "723cab76a5c14d61bb8fffc84a3ea699" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 40739 } } }
I20260808 23:26:48.728080 11064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 723cab76a5c14d61bb8fffc84a3ea699 [sys.catalog]: This master's current role is: LEADER
I20260808 23:26:48.728129 11063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 723cab76a5c14d61bb8fffc84a3ea699 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "723cab76a5c14d61bb8fffc84a3ea699" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "723cab76a5c14d61bb8fffc84a3ea699" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 40739 } } }
I20260808 23:26:48.728435 11063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 723cab76a5c14d61bb8fffc84a3ea699 [sys.catalog]: This master's current role is: LEADER
I20260808 23:26:48.736194 11067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:26:48.740617 11067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:26:48.748823 11067 catalog_manager.cc:1347] Generated new cluster ID: dc5dac64ce6948e89650dc7ee8d714ae
I20260808 23:26:48.748924 11067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:26:48.766218 11067 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:26:48.767910 11067 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:26:48.783591 11067 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 723cab76a5c14d61bb8fffc84a3ea699: Generated new TSK 0
I20260808 23:26:48.784305 11067 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.44.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
I20260808 23:26:48.837054 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:0
--local_ip_for_outbound_sockets=127.9.44.65
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:40739
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:45445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:26:49.151047 11090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:26:49.151319 11090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:26:49.151455 11090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:26:49.158298 11090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:26:49.158401 11090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:26:49.158520 11090 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 23:26:49.158628 11090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:26:49.167598 11090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:45445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.44.126:40739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:26:49.168674 11090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:26:49.169874 11090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:26:49.181802 11097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:49.181885 11096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:49.181879 11099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:26:49.182888 11090 server_base.cc:1034] running on GCE node
I20260808 23:26:49.183436 11090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:26:49.184265 11090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:26:49.185467 11090 hybrid_clock.cc:648] HybridClock initialized: now 1786231609185402 us; error 40 us; skew 500 ppm
I20260808 23:26:49.185709 11090 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:26:49 dist-test-slave-6bbx krb5kdc[10927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231609, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:26:49.188620 11090 init.cc:377] Logged in from keytab as kudu/127.9.44.65@KRBTEST.COM (short username kudu)
I20260808 23:26:49.189929 11090 webserver.cc:460] Webserver started at http://127.9.44.65:34049/ using document root <none> and password file <none>
I20260808 23:26:49.190551 11090 fs_manager.cc:359] Metadata directory not provided
I20260808 23:26:49.190652 11090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:26:49.190956 11090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:26:49.193660 11090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-0/data/instance:
uuid: "456a4efb580f440c82b49a9ab397863f"
format_stamp: "Formatted at 2026-08-08 23:26:49 on dist-test-slave-6bbx"
server_key: "e4dbf0b0c53db056ccd41712b17f90e9"
server_key_iv: "8f3a3da1c34912a085f9767ee0ea5d50"
server_key_version: "encryptionkey@0"
I20260808 23:26:49.194407 11090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance:
uuid: "456a4efb580f440c82b49a9ab397863f"
format_stamp: "Formatted at 2026-08-08 23:26:49 on dist-test-slave-6bbx"
server_key: "e4dbf0b0c53db056ccd41712b17f90e9"
server_key_iv: "8f3a3da1c34912a085f9767ee0ea5d50"
server_key_version: "encryptionkey@0"
I20260808 23:26:49.199179 11090 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 23:26:49.202461 11110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:49.203735 11090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:26:49.203881 11090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "456a4efb580f440c82b49a9ab397863f"
format_stamp: "Formatted at 2026-08-08 23:26:49 on dist-test-slave-6bbx"
server_key: "e4dbf0b0c53db056ccd41712b17f90e9"
server_key_iv: "8f3a3da1c34912a085f9767ee0ea5d50"
server_key_version: "encryptionkey@0"
I20260808 23:26:49.204077 11090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:26:49.219338 11090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:26:49.223202 11090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:26:49.223438 11090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:26:49.226034 11090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:26:49.226118 11090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:49.226260 11090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:26:49.226322 11090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 23:26:49 dist-test-slave-6bbx krb5kdc[10927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231609, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:26:49.248632 11125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:49.226625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:55037 (local address 127.9.44.126:40739)
0808 23:26:49.228718 (+ 2093us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:49.228724 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:49.228746 (+ 22us) server_negotiation.cc:407] Connection header received
0808 23:26:49.230344 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:49.230350 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:49.230436 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:49.230547 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:49.232579 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:49.233277 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:49.235750 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:49.236341 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:49.236443 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:49.241309 (+ 4866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:49.241318 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:49.241322 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:49.241366 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:49.244880 (+ 3514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:49.245797 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:49.245804 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:49.245808 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:49.245980 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:49.246608 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:49.246614 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:49.246617 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:49.246935 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:49.247087 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:49.248038 (+ 951us) server_negotiation.cc:299] Negotiation successful
0808 23:26:49.248273 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1962,"thread_start_us":1851,"threads_started":1}
I20260808 23:26:49.250101 11126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:49.227083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:40739 (local address 127.9.44.65:55037)
0808 23:26:49.227987 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:49.228029 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:49.229936 (+ 1907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:49.230825 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:49.230837 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:49.231554 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:26:49.232405 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:49.232428 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:49.233442 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:49.233449 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:49.235544 (+ 2095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:49.235560 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:49.236792 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:49.236801 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:49.237114 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:49.237831 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:26:49.237865 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:26:49.241060 (+ 3195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:26:49.245036 (+ 3976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:49.245045 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:49.245063 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:49.245638 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:49.246135 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:49.246141 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:49.246145 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:49.246461 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:49.247072 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:26:49.247079 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:26:49.247533 (+ 454us) client_negotiation.cc:769] Sending connection context
0808 23:26:49.247932 (+ 399us) client_negotiation.cc:240] Negotiation successful
0808 23:26:49.248470 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":341,"threads_started":1}
I20260808 23:26:49.267057 11090 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:45443
I20260808 23:26:49.267068 11279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:45443 every 8 connection(s)
I20260808 23:26:49.268319 11090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:26:49.269927 11236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:49.256214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:40739 (local address 127.9.44.65:47157)
0808 23:26:49.256752 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:49.256768 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:49.256897 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:49.257233 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:49.257238 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:49.257928 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:26:49.258477 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:49.258493 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:49.259537 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:49.259543 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:49.261459 (+ 1916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:49.261482 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:49.262511 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:49.262519 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:49.262701 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:49.263303 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:26:49.263326 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:26:49.264487 (+ 1161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:26:49.267417 (+ 2930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:49.267423 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:49.267428 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:49.267866 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:49.268228 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:49.268233 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:49.268237 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:49.268360 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:49.268880 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:26:49.268885 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:26:49.269106 (+ 221us) client_negotiation.cc:769] Sending connection context
0808 23:26:49.269260 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 23:26:49.269562 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":286,"threads_started":1}
I20260808 23:26:49.270649 11280 heartbeater.cc:347] Connected to a master server at 127.9.44.126:40739
I20260808 23:26:49.270723 11125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:49.256056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:47157 (local address 127.9.44.126:40739)
0808 23:26:49.256209 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:49.256213 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:49.256852 (+ 639us) server_negotiation.cc:407] Connection header received
0808 23:26:49.256999 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:49.257005 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:49.257078 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:49.257170 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:49.258662 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:49.259363 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:49.261615 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:49.262148 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:49.262267 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:49.264725 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:49.264731 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:49.264736 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:49.264765 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:49.267295 (+ 2530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:49.267972 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:49.267978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:49.267982 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:49.268111 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:49.268504 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:49.268507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:49.268509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:49.268766 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:49.268893 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:49.270214 (+ 1321us) server_negotiation.cc:299] Negotiation successful
0808 23:26:49.270416 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 23:26:49.270871 11280 heartbeater.cc:464] Registering TS with master...
I20260808 23:26:49.271713 11280 heartbeater.cc:511] Master 127.9.44.126:40739 requested a full tablet report, sending...
I20260808 23:26:49.272918 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 11090
I20260808 23:26:49.273152 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance
I20260808 23:26:49.273965 9393 external_mini_cluster.cc:1428] Setting key cef1da9aef179a7ce6fe3d389b55bac3
I20260808 23:26:49.274062 10987 ts_manager.cc:194] Registered new tserver with Master: 456a4efb580f440c82b49a9ab397863f (127.9.44.65:45443)
I20260808 23:26:49.275710 10987 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.65@KRBTEST.COM'} at 127.9.44.65:55037
WARNING: no policy specified for kudu/127.9.44.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
I20260808 23:26:49.312855 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:0
--local_ip_for_outbound_sockets=127.9.44.66
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:40739
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:45445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:26:49.620643 11286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:26:49.620918 11286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:26:49.621053 11286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:26:49.628171 11286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:26:49.628276 11286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:26:49.628394 11286 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 23:26:49.628501 11286 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:26:49.637779 11286 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:45445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.44.126:40739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:26:49.638909 11286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:26:49.640177 11286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:26:49.652180 11294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:49.652225 11293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:49.652262 11296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:26:49.653347 11286 server_base.cc:1034] running on GCE node
I20260808 23:26:49.653908 11286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:26:49.654712 11286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:26:49.655910 11286 hybrid_clock.cc:648] HybridClock initialized: now 1786231609655835 us; error 51 us; skew 500 ppm
I20260808 23:26:49.656169 11286 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:26:49 dist-test-slave-6bbx krb5kdc[10927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231609, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:26:49.659083 11286 init.cc:377] Logged in from keytab as kudu/127.9.44.66@KRBTEST.COM (short username kudu)
I20260808 23:26:49.660442 11286 webserver.cc:460] Webserver started at http://127.9.44.66:35217/ using document root <none> and password file <none>
I20260808 23:26:49.661060 11286 fs_manager.cc:359] Metadata directory not provided
I20260808 23:26:49.661144 11286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:26:49.661419 11286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:26:49.664160 11286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-1/data/instance:
uuid: "b24abb45c55d4160b24f641734a5e793"
format_stamp: "Formatted at 2026-08-08 23:26:49 on dist-test-slave-6bbx"
server_key: "711b463362eb811d30f397cd91e91a90"
server_key_iv: "3da6cd59765152d39c07ce96abf3fd85"
server_key_version: "encryptionkey@0"
I20260808 23:26:49.664885 11286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance:
uuid: "b24abb45c55d4160b24f641734a5e793"
format_stamp: "Formatted at 2026-08-08 23:26:49 on dist-test-slave-6bbx"
server_key: "711b463362eb811d30f397cd91e91a90"
server_key_iv: "3da6cd59765152d39c07ce96abf3fd85"
server_key_version: "encryptionkey@0"
I20260808 23:26:49.669608 11286 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 23:26:49.672792 11306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:49.674001 11286 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:26:49.674134 11286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "b24abb45c55d4160b24f641734a5e793"
format_stamp: "Formatted at 2026-08-08 23:26:49 on dist-test-slave-6bbx"
server_key: "711b463362eb811d30f397cd91e91a90"
server_key_iv: "3da6cd59765152d39c07ce96abf3fd85"
server_key_version: "encryptionkey@0"
I20260808 23:26:49.674304 11286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:26:49.701347 11286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:26:49.705165 11286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:26:49.705385 11286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:26:49.707834 11286 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:26:49.707916 11286 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:49.708067 11286 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:26:49.708128 11286 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 23:26:49 dist-test-slave-6bbx krb5kdc[10927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231609, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:26:49.729575 11125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:49.708501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:57661 (local address 127.9.44.126:40739)
0808 23:26:49.708725 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:49.708731 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:49.710118 (+ 1387us) server_negotiation.cc:407] Connection header received
0808 23:26:49.712831 (+ 2713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:49.712838 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:49.712950 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:49.713032 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:49.715694 (+ 2662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:49.716568 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:49.718381 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:49.718778 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:49.718888 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:49.722941 (+ 4053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:49.722947 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:49.722952 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:49.722994 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:49.726105 (+ 3111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:49.726873 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:49.726879 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:49.726883 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:49.727059 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:49.727723 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:49.727728 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:49.727731 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:49.728029 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:49.728188 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:49.729016 (+ 828us) server_negotiation.cc:299] Negotiation successful
0808 23:26:49.729250 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 23:26:49.730472 11318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:49.709053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:40739 (local address 127.9.44.66:57661)
0808 23:26:49.709936 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:49.709996 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:49.712468 (+ 2472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:49.713575 (+ 1107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:49.713590 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:49.714424 (+ 834us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:26:49.715444 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:49.715478 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:49.716744 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:49.716748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:49.718241 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:49.718254 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:49.719024 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:49.719033 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:49.719246 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:49.719860 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:26:49.719885 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:26:49.722728 (+ 2843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:26:49.726302 (+ 3574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:49.726315 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:49.726347 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:49.726759 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:49.727189 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:49.727195 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:49.727200 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:49.727527 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:49.728281 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:26:49.728289 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:26:49.728668 (+ 379us) client_negotiation.cc:769] Sending connection context
0808 23:26:49.728902 (+ 234us) client_negotiation.cc:240] Negotiation successful
0808 23:26:49.729363 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":338,"threads_started":1}
I20260808 23:26:49.749631 11286 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:42971
I20260808 23:26:49.749693 11476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:42971 every 8 connection(s)
I20260808 23:26:49.750924 11286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:26:49.753154 11477 heartbeater.cc:347] Connected to a master server at 127.9.44.126:40739
I20260808 23:26:49.753443 11477 heartbeater.cc:464] Registering TS with master...
I20260808 23:26:49.754215 11477 heartbeater.cc:511] Master 127.9.44.126:40739 requested a full tablet report, sending...
I20260808 23:26:49.754891 11441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:49.739856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:40739 (local address 127.9.44.66:41453)
0808 23:26:49.740682 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:49.740697 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:49.740831 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:49.741159 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:49.741163 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:49.741790 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:26:49.742063 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:49.742072 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:49.743445 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:49.743451 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:49.745259 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:49.745280 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:49.746462 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:49.746468 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:49.746639 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:49.747355 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:26:49.747381 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:26:49.748241 (+ 860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:26:49.751501 (+ 3260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:49.751506 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:49.751510 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:49.752075 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:49.753089 (+ 1014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:49.753094 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:49.753097 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:49.753266 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:49.753974 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:26:49.753977 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:26:49.754187 (+ 210us) client_negotiation.cc:769] Sending connection context
0808 23:26:49.754316 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 23:26:49.754561 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"mutex_wait_us":3,"thread_start_us":576,"threads_started":1}
I20260808 23:26:49.755028 11125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:49.736304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:41453 (local address 127.9.44.126:40739)
0808 23:26:49.736464 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:49.736470 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:49.740779 (+ 4309us) server_negotiation.cc:407] Connection header received
0808 23:26:49.740952 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:49.740958 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:49.741031 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:49.741107 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:49.742215 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:49.743258 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:49.745453 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:49.746067 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:49.746200 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:49.748420 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:49.748426 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:49.748431 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:49.748465 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:49.751319 (+ 2854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:49.752296 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:49.752301 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:49.752305 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:49.752431 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:49.753407 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:49.753411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:49.753414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:49.753858 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:49.754008 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:49.754462 (+ 454us) server_negotiation.cc:299] Negotiation successful
0808 23:26:49.754710 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 23:26:49.755759 10987 ts_manager.cc:194] Registered new tserver with Master: b24abb45c55d4160b24f641734a5e793 (127.9.44.66:42971)
I20260808 23:26:49.756611 10987 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.66@KRBTEST.COM'} at 127.9.44.66:57661
I20260808 23:26:49.756939 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 11286
I20260808 23:26:49.757121 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance
I20260808 23:26:49.757644 9393 external_mini_cluster.cc:1428] Setting key 5b316c1948c1ab371ad9bde7bbc330ba
WARNING: no policy specified for kudu/127.9.44.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
I20260808 23:26:49.795795 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:0
--local_ip_for_outbound_sockets=127.9.44.67
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:40739
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:45445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:26:50.107409 11481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:26:50.107715 11481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:26:50.107831 11481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:26:50.114665 11481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:26:50.114750 11481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:26:50.114835 11481 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 23:26:50.114912 11481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:26:50.123847 11481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:45445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.44.126:40739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:26:50.124886 11481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:26:50.126114 11481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:26:50.138047 11490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:50.138084 11486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:50.138124 11488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:26:50.138931 11481 server_base.cc:1034] running on GCE node
I20260808 23:26:50.139504 11481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:26:50.140341 11481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:26:50.141546 11481 hybrid_clock.cc:648] HybridClock initialized: now 1786231610141481 us; error 35 us; skew 500 ppm
I20260808 23:26:50.141810 11481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:26:50 dist-test-slave-6bbx krb5kdc[10927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231610, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:26:50.144702 11481 init.cc:377] Logged in from keytab as kudu/127.9.44.67@KRBTEST.COM (short username kudu)
I20260808 23:26:50.145958 11481 webserver.cc:460] Webserver started at http://127.9.44.67:36245/ using document root <none> and password file <none>
I20260808 23:26:50.146534 11481 fs_manager.cc:359] Metadata directory not provided
I20260808 23:26:50.146615 11481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:26:50.146883 11481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:26:50.149587 11481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-2/data/instance:
uuid: "ee373c494f1e45a8b19b0c545a924200"
format_stamp: "Formatted at 2026-08-08 23:26:50 on dist-test-slave-6bbx"
server_key: "93888566232429384f71ad4b297acd5b"
server_key_iv: "56e349d294cfaf320bb54453498a61da"
server_key_version: "encryptionkey@0"
I20260808 23:26:50.150290 11481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance:
uuid: "ee373c494f1e45a8b19b0c545a924200"
format_stamp: "Formatted at 2026-08-08 23:26:50 on dist-test-slave-6bbx"
server_key: "93888566232429384f71ad4b297acd5b"
server_key_iv: "56e349d294cfaf320bb54453498a61da"
server_key_version: "encryptionkey@0"
I20260808 23:26:50.155028 11481 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 23:26:50.158159 11499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:50.159301 11481 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 23:26:50.159439 11481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "ee373c494f1e45a8b19b0c545a924200"
format_stamp: "Formatted at 2026-08-08 23:26:50 on dist-test-slave-6bbx"
server_key: "93888566232429384f71ad4b297acd5b"
server_key_iv: "56e349d294cfaf320bb54453498a61da"
server_key_version: "encryptionkey@0"
I20260808 23:26:50.159632 11481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:26:50.180945 11481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:26:50.184309 11481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:26:50.184530 11481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:26:50.186939 11481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:26:50.187073 11481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260808 23:26:50.187212 11481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:26:50.187278 11481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 23:26:50 dist-test-slave-6bbx krb5kdc[10927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231610, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:26:50.208460 11125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:50.187233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:35621 (local address 127.9.44.126:40739)
0808 23:26:50.187476 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:50.187483 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:50.188825 (+ 1342us) server_negotiation.cc:407] Connection header received
0808 23:26:50.191387 (+ 2562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:50.191392 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:50.191476 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:50.191589 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:50.193833 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.194777 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.197058 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.197572 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.197696 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:50.201433 (+ 3737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:50.201438 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:50.201441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:50.201475 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:50.204550 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:50.205556 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:50.205562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:50.205566 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:50.205742 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:50.206347 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:50.206352 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:50.206355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:50.206650 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:50.206810 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:50.207914 (+ 1104us) server_negotiation.cc:299] Negotiation successful
0808 23:26:50.208082 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 23:26:50.209564 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:50.187794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:40739 (local address 127.9.44.67:35621)
0808 23:26:50.188662 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:50.188708 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:50.191096 (+ 2388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:50.191888 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:50.191900 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:50.192678 (+ 778us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:26:50.193623 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:50.193654 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.194959 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.194963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:50.196916 (+ 1953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:50.196933 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.197806 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.197815 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:50.198076 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:50.198782 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:26:50.198820 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:26:50.201227 (+ 2407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:26:50.204684 (+ 3457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:50.204694 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:50.204708 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:50.205399 (+ 691us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:50.205907 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:50.205913 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:50.205917 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:50.206202 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:50.206806 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:26:50.206815 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:26:50.207328 (+ 513us) client_negotiation.cc:769] Sending connection context
0808 23:26:50.207659 (+ 331us) client_negotiation.cc:240] Negotiation successful
0808 23:26:50.208200 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":337,"threads_started":1}
I20260808 23:26:50.228152 11125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:50.215287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:45147 (local address 127.9.44.126:40739)
0808 23:26:50.215460 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:50.215466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:50.216133 (+ 667us) server_negotiation.cc:407] Connection header received
0808 23:26:50.216369 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:50.216373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:50.216444 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:50.216535 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:50.217798 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.218727 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.220118 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.220589 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.220696 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:50.222749 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:50.222754 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:50.222757 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:50.222784 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:50.225545 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:50.226359 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:50.226364 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:50.226367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:50.226499 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:50.226828 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:50.226833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:50.226837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:50.227071 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:50.227195 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:50.227607 (+ 412us) server_negotiation.cc:299] Negotiation successful
0808 23:26:50.227829 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 23:26:50.228269 11600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:50.215285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:40739 (local address 127.9.44.67:45147)
0808 23:26:50.216015 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:50.216033 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:50.216195 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:50.216564 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:50.216569 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:50.217176 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:26:50.217654 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:50.217670 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.218853 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.218859 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:50.219936 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:50.219952 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.220774 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.220781 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:50.220961 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:50.221571 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:26:50.221594 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:26:50.222575 (+ 981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:26:50.225674 (+ 3099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:50.225682 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:50.225686 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:50.226240 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:50.226601 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:50.226606 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:50.226610 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:50.226730 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:50.227179 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:26:50.227184 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:26:50.227357 (+ 173us) client_negotiation.cc:769] Sending connection context
0808 23:26:50.227482 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 23:26:50.227802 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":425,"threads_started":1}
I20260808 23:26:50.233721 11481 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:39141
I20260808 23:26:50.233742 11667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:39141 every 8 connection(s)
I20260808 23:26:50.234817 11481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:26:50.236505 11668 heartbeater.cc:347] Connected to a master server at 127.9.44.126:40739
I20260808 23:26:50.236712 11668 heartbeater.cc:464] Registering TS with master...
I20260808 23:26:50.237279 11668 heartbeater.cc:511] Master 127.9.44.126:40739 requested a full tablet report, sending...
I20260808 23:26:50.238488 10987 ts_manager.cc:194] Registered new tserver with Master: ee373c494f1e45a8b19b0c545a924200 (127.9.44.67:39141)
I20260808 23:26:50.239279 10987 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.67@KRBTEST.COM'} at 127.9.44.67:35621
I20260808 23:26:50.240551 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 11481
I20260808 23:26:50.240731 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance
I20260808 23:26:50.241142 9393 external_mini_cluster.cc:1428] Setting key b9a2af4c090e0312655b87610350e771
I20260808 23:26:50.243907 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 23:26:50 dist-test-slave-6bbx krb5kdc[10927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231610, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
I20260808 23:26:50.278512 11280 heartbeater.cc:503] Master 127.9.44.126:40739 was elected leader, sending a full tablet report...
Aug 08 23:26:50 dist-test-slave-6bbx krb5kdc[10927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231610, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:26:50.291531 11125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:50.277959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38550 (local address 127.9.44.126:40739)
0808 23:26:50.278168 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:50.278175 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:50.278440 (+ 265us) server_negotiation.cc:407] Connection header received
0808 23:26:50.278629 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:50.278634 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:50.278727 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:50.278813 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:50.280180 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.280915 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.282225 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.282689 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.282889 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:50.286505 (+ 3616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:50.286510 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:50.286514 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:50.286544 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:50.289079 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:50.289704 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:50.289708 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:50.289710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:50.289831 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:50.290186 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:50.290191 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:50.290195 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:50.290467 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:50.290700 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:50.290985 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 23:26:50.291198 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 23:26:50.295125 10987 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38550:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:26:50.301945 10987 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:26:50.334368 11687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:50.313389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57284 (local address 127.9.44.66:42971)
0808 23:26:50.325755 (+ 12366us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:50.325764 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:50.325794 (+ 30us) server_negotiation.cc:407] Connection header received
0808 23:26:50.325894 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:50.325902 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:50.326145 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:50.326401 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:26:50.327404 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.328899 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.331970 (+ 3071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.333206 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.333330 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:50.333485 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:50.333930 (+ 445us) server_negotiation.cc:299] Negotiation successful
0808 23:26:50.334087 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12186,"thread_start_us":12052,"threads_started":1}
I20260808 23:26:50.335466 11688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:50.313225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:42971 (local address 127.0.0.1:57284)
0808 23:26:50.324342 (+ 11117us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:50.324389 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:50.324551 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:50.326661 (+ 2110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:50.326668 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:50.326701 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:26:50.327103 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:50.327114 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.329044 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.329049 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:50.331791 (+ 2742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:50.331810 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.333599 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.333603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:50.333732 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:50.333764 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 23:26:50.333885 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 23:26:50.334056 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10858,"mutex_wait_us":56,"thread_start_us":12540,"threads_started":1}
I20260808 23:26:50.335990 11692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:50.314104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46526 (local address 127.9.44.65:45443)
0808 23:26:50.314607 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:50.314616 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:50.326308 (+ 11692us) server_negotiation.cc:407] Connection header received
0808 23:26:50.326418 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:50.326426 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:50.326652 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:50.326844 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:26:50.327772 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.329345 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.332658 (+ 3313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.334223 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.334367 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:50.334610 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:50.335404 (+ 794us) server_negotiation.cc:299] Negotiation successful
0808 23:26:50.335607 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":236,"threads_started":1}
I20260808 23:26:50.337647 11685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:50.313539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:45443 (local address 127.0.0.1:46526)
0808 23:26:50.326042 (+ 12503us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:50.326060 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:50.326185 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:50.327077 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:50.327083 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:50.327110 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:26:50.327555 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:50.327566 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.329547 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.329553 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:50.332467 (+ 2914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:50.332482 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.334999 (+ 2517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.335011 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:50.335213 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:50.335245 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 23:26:50.335335 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 23:26:50.335463 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12374,"mutex_wait_us":513,"thread_start_us":11202,"threads_started":1}
I20260808 23:26:50.339287 11374 tablet_service.cc:1467] Processing CreateTablet for tablet 0901be26007f4d708274370790c0f0dc (DEFAULT_TABLE table=test-table [id=239b2362dbad4e8eb6a2f428e8cf9d52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:26:50.340359 11693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:50.315516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:39141 (local address 127.0.0.1:41732)
0808 23:26:50.330785 (+ 15269us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:50.330798 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:50.330919 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:50.331385 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:50.331389 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:50.331411 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:26:50.331867 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:50.331881 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.334292 (+ 2411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.334297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:50.337485 (+ 3188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:50.337498 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.339595 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.339602 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:50.339811 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:50.339845 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 23:26:50.339913 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 23:26:50.340073 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15153,"thread_start_us":14827,"threads_started":1}
I20260808 23:26:50.340528 11694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:50.316068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41732 (local address 127.9.44.67:39141)
0808 23:26:50.330099 (+ 14031us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:50.330104 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:50.330894 (+ 790us) server_negotiation.cc:407] Connection header received
0808 23:26:50.331050 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:50.331055 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:50.331221 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:50.331374 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:26:50.332212 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.334030 (+ 1818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.337642 (+ 3612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.339387 (+ 1745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.339524 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:50.339920 (+ 396us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:50.340019 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 23:26:50.340219 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13902,"thread_start_us":13741,"threads_started":1}
I20260808 23:26:50.340898 11374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0901be26007f4d708274370790c0f0dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:50.340664 11182 tablet_service.cc:1467] Processing CreateTablet for tablet 0901be26007f4d708274370790c0f0dc (DEFAULT_TABLE table=test-table [id=239b2362dbad4e8eb6a2f428e8cf9d52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:26:50.341964 11182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0901be26007f4d708274370790c0f0dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:50.344229 11563 tablet_service.cc:1467] Processing CreateTablet for tablet 0901be26007f4d708274370790c0f0dc (DEFAULT_TABLE table=test-table [id=239b2362dbad4e8eb6a2f428e8cf9d52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:26:50.345515 11563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0901be26007f4d708274370790c0f0dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:50.351818 11695 tablet_bootstrap.cc:492] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793: Bootstrap starting.
I20260808 23:26:50.354786 11695 tablet_bootstrap.cc:654] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:50.355784 11695 log.cc:824] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793: Log is configured to *not* fsync() on all Append() calls
I20260808 23:26:50.356410 11696 tablet_bootstrap.cc:492] T 0901be26007f4d708274370790c0f0dc P 456a4efb580f440c82b49a9ab397863f: Bootstrap starting.
I20260808 23:26:50.359118 11700 tablet_bootstrap.cc:492] T 0901be26007f4d708274370790c0f0dc P ee373c494f1e45a8b19b0c545a924200: Bootstrap starting.
I20260808 23:26:50.359498 11695 tablet_bootstrap.cc:492] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793: No bootstrap required, opened a new log
I20260808 23:26:50.359644 11696 tablet_bootstrap.cc:654] T 0901be26007f4d708274370790c0f0dc P 456a4efb580f440c82b49a9ab397863f: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:50.359848 11695 ts_tablet_manager.cc:1397] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 23:26:50.361035 11696 log.cc:824] T 0901be26007f4d708274370790c0f0dc P 456a4efb580f440c82b49a9ab397863f: Log is configured to *not* fsync() on all Append() calls
I20260808 23:26:50.362952 11700 tablet_bootstrap.cc:654] T 0901be26007f4d708274370790c0f0dc P ee373c494f1e45a8b19b0c545a924200: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:50.364651 11700 log.cc:824] T 0901be26007f4d708274370790c0f0dc P ee373c494f1e45a8b19b0c545a924200: Log is configured to *not* fsync() on all Append() calls
I20260808 23:26:50.364835 11696 tablet_bootstrap.cc:492] T 0901be26007f4d708274370790c0f0dc P 456a4efb580f440c82b49a9ab397863f: No bootstrap required, opened a new log
I20260808 23:26:50.365084 11696 ts_tablet_manager.cc:1397] T 0901be26007f4d708274370790c0f0dc P 456a4efb580f440c82b49a9ab397863f: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.005s
I20260808 23:26:50.367744 11700 tablet_bootstrap.cc:492] T 0901be26007f4d708274370790c0f0dc P ee373c494f1e45a8b19b0c545a924200: No bootstrap required, opened a new log
I20260808 23:26:50.367973 11700 ts_tablet_manager.cc:1397] T 0901be26007f4d708274370790c0f0dc P ee373c494f1e45a8b19b0c545a924200: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.004s
I20260808 23:26:50.373023 11695 raft_consensus.cc:348] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.373579 11695 raft_consensus.cc:374] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:50.373731 11695 raft_consensus.cc:729] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24abb45c55d4160b24f641734a5e793, State: Initialized, Role: FOLLOWER
I20260808 23:26:50.374475 11695 consensus_queue.cc:260] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.375957 11477 heartbeater.cc:503] Master 127.9.44.126:40739 was elected leader, sending a full tablet report...
I20260808 23:26:50.376680 11695 ts_tablet_manager.cc:1428] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
I20260808 23:26:50.376904 11696 raft_consensus.cc:348] T 0901be26007f4d708274370790c0f0dc P 456a4efb580f440c82b49a9ab397863f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.377327 11696 raft_consensus.cc:374] T 0901be26007f4d708274370790c0f0dc P 456a4efb580f440c82b49a9ab397863f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:50.377454 11696 raft_consensus.cc:729] T 0901be26007f4d708274370790c0f0dc P 456a4efb580f440c82b49a9ab397863f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 456a4efb580f440c82b49a9ab397863f, State: Initialized, Role: FOLLOWER
I20260808 23:26:50.377954 11696 consensus_queue.cc:260] T 0901be26007f4d708274370790c0f0dc P 456a4efb580f440c82b49a9ab397863f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.379388 11700 raft_consensus.cc:348] T 0901be26007f4d708274370790c0f0dc P ee373c494f1e45a8b19b0c545a924200 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.380046 11700 raft_consensus.cc:374] T 0901be26007f4d708274370790c0f0dc P ee373c494f1e45a8b19b0c545a924200 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:50.380231 11696 ts_tablet_manager.cc:1428] T 0901be26007f4d708274370790c0f0dc P 456a4efb580f440c82b49a9ab397863f: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260808 23:26:50.380218 11700 raft_consensus.cc:729] T 0901be26007f4d708274370790c0f0dc P ee373c494f1e45a8b19b0c545a924200 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee373c494f1e45a8b19b0c545a924200, State: Initialized, Role: FOLLOWER
I20260808 23:26:50.380995 11700 consensus_queue.cc:260] T 0901be26007f4d708274370790c0f0dc P ee373c494f1e45a8b19b0c545a924200 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.382360 11668 heartbeater.cc:503] Master 127.9.44.126:40739 was elected leader, sending a full tablet report...
I20260808 23:26:50.382896 11700 ts_tablet_manager.cc:1428] T 0901be26007f4d708274370790c0f0dc P ee373c494f1e45a8b19b0c545a924200: Time spent starting tablet: real 0.015s user 0.009s sys 0.004s
I20260808 23:26:50.442390 11708 raft_consensus.cc:482] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:26:50.442691 11708 raft_consensus.cc:504] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.444305 11708 leader_election.cc:288] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 456a4efb580f440c82b49a9ab397863f (127.9.44.65:45443), ee373c494f1e45a8b19b0c545a924200 (127.9.44.67:39141)
I20260808 23:26:50.452728 11694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:50.444756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:40247 (local address 127.9.44.67:39141)
0808 23:26:50.444978 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:50.444985 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:50.446058 (+ 1073us) server_negotiation.cc:407] Connection header received
0808 23:26:50.446263 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:50.446267 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:50.446327 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:50.446402 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:26:50.446984 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.448051 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.450585 (+ 2534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.451486 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.451604 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:50.451744 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:50.452309 (+ 565us) server_negotiation.cc:299] Negotiation successful
0808 23:26:50.452472 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 23:26:50.453154 11713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:50.444581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:39141 (local address 127.9.44.66:40247)
0808 23:26:50.445946 (+ 1365us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:50.445966 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:50.446120 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:50.446437 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:50.446442 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:50.446469 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:26:50.446838 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:50.446848 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.448167 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.448173 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:50.450420 (+ 2247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:50.450433 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.451938 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.451944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:50.452120 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:50.452175 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 23:26:50.452273 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 23:26:50.452458 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1163,"mutex_wait_us":1,"thread_start_us":785,"threads_started":1}
I20260808 23:26:50.453379 11692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:50.444734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:40051 (local address 127.9.44.65:45443)
0808 23:26:50.444940 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:50.444947 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:50.446486 (+ 1539us) server_negotiation.cc:407] Connection header received
0808 23:26:50.446699 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:50.446703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:50.446782 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:50.446856 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:26:50.447647 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.448611 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.450853 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.452157 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.452316 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:50.452445 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:50.452777 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 23:26:50.453095 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 23:26:50.453542 11714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:50.444945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:45443 (local address 127.9.44.66:40051)
0808 23:26:50.446381 (+ 1436us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:50.446395 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:50.446564 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:50.446891 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:50.446895 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:50.446918 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:26:50.447479 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:50.447491 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.448780 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.448784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:50.450674 (+ 1890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:50.450687 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.452494 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.452501 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:50.452636 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:50.452659 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 23:26:50.452744 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 23:26:50.452864 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1292,"mutex_wait_us":22,"thread_start_us":1025,"threads_started":1}
I20260808 23:26:50.454293 11598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0901be26007f4d708274370790c0f0dc" candidate_uuid: "b24abb45c55d4160b24f641734a5e793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee373c494f1e45a8b19b0c545a924200" is_pre_election: true
I20260808 23:26:50.454828 11207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0901be26007f4d708274370790c0f0dc" candidate_uuid: "b24abb45c55d4160b24f641734a5e793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "456a4efb580f440c82b49a9ab397863f" is_pre_election: true
I20260808 23:26:50.455288 11598 raft_consensus.cc:2454] T 0901be26007f4d708274370790c0f0dc P ee373c494f1e45a8b19b0c545a924200 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b24abb45c55d4160b24f641734a5e793 in term 0.
I20260808 23:26:50.455576 11207 raft_consensus.cc:2454] T 0901be26007f4d708274370790c0f0dc P 456a4efb580f440c82b49a9ab397863f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b24abb45c55d4160b24f641734a5e793 in term 0.
I20260808 23:26:50.456238 11310 leader_election.cc:302] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24abb45c55d4160b24f641734a5e793, ee373c494f1e45a8b19b0c545a924200; no voters:
I20260808 23:26:50.456717 11708 raft_consensus.cc:2781] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:26:50.456919 11708 raft_consensus.cc:482] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:26:50.457129 11708 raft_consensus.cc:3037] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:50.459646 11708 raft_consensus.cc:504] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.460307 11708 leader_election.cc:288] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793 [CANDIDATE]: Term 1 election: Requested vote from peers 456a4efb580f440c82b49a9ab397863f (127.9.44.65:45443), ee373c494f1e45a8b19b0c545a924200 (127.9.44.67:39141)
I20260808 23:26:50.460757 11207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0901be26007f4d708274370790c0f0dc" candidate_uuid: "b24abb45c55d4160b24f641734a5e793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "456a4efb580f440c82b49a9ab397863f"
I20260808 23:26:50.460851 11598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0901be26007f4d708274370790c0f0dc" candidate_uuid: "b24abb45c55d4160b24f641734a5e793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee373c494f1e45a8b19b0c545a924200"
I20260808 23:26:50.461061 11598 raft_consensus.cc:3037] T 0901be26007f4d708274370790c0f0dc P ee373c494f1e45a8b19b0c545a924200 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:50.461061 11207 raft_consensus.cc:3037] T 0901be26007f4d708274370790c0f0dc P 456a4efb580f440c82b49a9ab397863f [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:50.464990 11207 raft_consensus.cc:2454] T 0901be26007f4d708274370790c0f0dc P 456a4efb580f440c82b49a9ab397863f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24abb45c55d4160b24f641734a5e793 in term 1.
I20260808 23:26:50.465132 11598 raft_consensus.cc:2454] T 0901be26007f4d708274370790c0f0dc P ee373c494f1e45a8b19b0c545a924200 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24abb45c55d4160b24f641734a5e793 in term 1.
I20260808 23:26:50.465473 11312 leader_election.cc:302] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 456a4efb580f440c82b49a9ab397863f, b24abb45c55d4160b24f641734a5e793; no voters:
I20260808 23:26:50.465867 11708 raft_consensus.cc:2781] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:26:50.466471 11708 raft_consensus.cc:686] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793 [term 1 LEADER]: Becoming Leader. State: Replica: b24abb45c55d4160b24f641734a5e793, State: Running, Role: LEADER
I20260808 23:26:50.466928 11708 consensus_queue.cc:237] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.471539 10987 catalog_manager.cc:5495] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793 reported cstate change: term changed from 0 to 1, leader changed from <none> to b24abb45c55d4160b24f641734a5e793 (127.9.44.66). New cstate: current_term: 1 leader_uuid: "b24abb45c55d4160b24f641734a5e793" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } health_report { overall_health: UNKNOWN } } }
W20260808 23:26:50.486223 11669 tablet.cc:2369] T 0901be26007f4d708274370790c0f0dc P ee373c494f1e45a8b19b0c545a924200: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:26:50.503844 11478 tablet.cc:2369] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:26:50.520582 11281 tablet.cc:2369] T 0901be26007f4d708274370790c0f0dc P 456a4efb580f440c82b49a9ab397863f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:26:50.522264 11687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:50.517004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57300 (local address 127.9.44.66:42971)
0808 23:26:50.517167 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:50.517171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:50.517247 (+ 76us) server_negotiation.cc:407] Connection header received
0808 23:26:50.517543 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:50.517546 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:50.517618 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:50.517735 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:26:50.518019 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.518717 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.520089 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.520543 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.520639 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:50.521210 (+ 571us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:26:50.521722 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:26:50.521834 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:50.521944 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 23:26:50.522010 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 23:26:50.531255 11598 raft_consensus.cc:1261] T 0901be26007f4d708274370790c0f0dc P ee373c494f1e45a8b19b0c545a924200 [term 1 FOLLOWER]: Refusing update from remote peer b24abb45c55d4160b24f641734a5e793: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:26:50.532418 11715 consensus_queue.cc:1035] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:26:50.532950 11207 raft_consensus.cc:1261] T 0901be26007f4d708274370790c0f0dc P 456a4efb580f440c82b49a9ab397863f [term 1 FOLLOWER]: Refusing update from remote peer b24abb45c55d4160b24f641734a5e793: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:26:50.533829 11715 consensus_queue.cc:1035] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793 [LEADER]: Connected to new peer: Peer: permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:26:50.568670 10987 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:26:50.568971 10987 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:26:50.572587 10987 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 723cab76a5c14d61bb8fffc84a3ea699: Sending DeleteTablet for 3 replicas of tablet 0901be26007f4d708274370790c0f0dc
I20260808 23:26:50.573927 11182 tablet_service.cc:1514] Processing DeleteTablet for tablet 0901be26007f4d708274370790c0f0dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:26:50 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:46526
I20260808 23:26:50.573935 11374 tablet_service.cc:1514] Processing DeleteTablet for tablet 0901be26007f4d708274370790c0f0dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:26:50 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:57284
I20260808 23:26:50.574201 11563 tablet_service.cc:1514] Processing DeleteTablet for tablet 0901be26007f4d708274370790c0f0dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:26:50 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:41732
I20260808 23:26:50.580298 10985 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38550:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:26:50.581761 10985 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:26:50.586751 11735 tablet_replica.cc:330] T 0901be26007f4d708274370790c0f0dc P ee373c494f1e45a8b19b0c545a924200: stopping tablet replica
I20260808 23:26:50.587328 11735 raft_consensus.cc:2229] T 0901be26007f4d708274370790c0f0dc P ee373c494f1e45a8b19b0c545a924200 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:26:50.587839 11735 raft_consensus.cc:2258] T 0901be26007f4d708274370790c0f0dc P ee373c494f1e45a8b19b0c545a924200 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:26:50.587991 11733 tablet_replica.cc:330] T 0901be26007f4d708274370790c0f0dc P 456a4efb580f440c82b49a9ab397863f: stopping tablet replica
I20260808 23:26:50.588654 11733 raft_consensus.cc:2229] T 0901be26007f4d708274370790c0f0dc P 456a4efb580f440c82b49a9ab397863f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:26:50.588827 11734 tablet_replica.cc:330] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793: stopping tablet replica
I20260808 23:26:50.589728 11374 tablet_service.cc:1467] Processing CreateTablet for tablet 5b363fa7f3a8413c93fa538e608ee72d (DEFAULT_TABLE table=test-table [id=33b9d7e3abaf465d813b9ec83c6621f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:26:50.589936 11563 tablet_service.cc:1467] Processing CreateTablet for tablet 5b363fa7f3a8413c93fa538e608ee72d (DEFAULT_TABLE table=test-table [id=33b9d7e3abaf465d813b9ec83c6621f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:26:50.590039 11182 tablet_service.cc:1467] Processing CreateTablet for tablet 5b363fa7f3a8413c93fa538e608ee72d (DEFAULT_TABLE table=test-table [id=33b9d7e3abaf465d813b9ec83c6621f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:26:50.590310 11374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b363fa7f3a8413c93fa538e608ee72d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:50.590446 11563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b363fa7f3a8413c93fa538e608ee72d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:50.590677 11182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b363fa7f3a8413c93fa538e608ee72d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:50.592435 11734 raft_consensus.cc:2229] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:26:50.593758 11734 raft_consensus.cc:2258] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:26:50.594328 11733 raft_consensus.cc:2258] T 0901be26007f4d708274370790c0f0dc P 456a4efb580f440c82b49a9ab397863f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:26:50.596895 11735 ts_tablet_manager.cc:1905] T 0901be26007f4d708274370790c0f0dc P ee373c494f1e45a8b19b0c545a924200: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:26:50.597775 11733 ts_tablet_manager.cc:1905] T 0901be26007f4d708274370790c0f0dc P 456a4efb580f440c82b49a9ab397863f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:26:50.597981 11696 tablet_bootstrap.cc:492] T 5b363fa7f3a8413c93fa538e608ee72d P 456a4efb580f440c82b49a9ab397863f: Bootstrap starting.
I20260808 23:26:50.600010 11696 tablet_bootstrap.cc:654] T 5b363fa7f3a8413c93fa538e608ee72d P 456a4efb580f440c82b49a9ab397863f: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:50.601168 11700 tablet_bootstrap.cc:492] T 5b363fa7f3a8413c93fa538e608ee72d P ee373c494f1e45a8b19b0c545a924200: Bootstrap starting.
I20260808 23:26:50.603271 11700 tablet_bootstrap.cc:654] T 5b363fa7f3a8413c93fa538e608ee72d P ee373c494f1e45a8b19b0c545a924200: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:50.603518 11695 tablet_bootstrap.cc:492] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793: Bootstrap starting.
I20260808 23:26:50.604619 11735 ts_tablet_manager.cc:1918] T 0901be26007f4d708274370790c0f0dc P ee373c494f1e45a8b19b0c545a924200: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:26:50.604844 11735 log.cc:1196] T 0901be26007f4d708274370790c0f0dc P ee373c494f1e45a8b19b0c545a924200: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/0901be26007f4d708274370790c0f0dc
I20260808 23:26:50.606293 11735 ts_tablet_manager.cc:1939] T 0901be26007f4d708274370790c0f0dc P ee373c494f1e45a8b19b0c545a924200: Deleting consensus metadata
I20260808 23:26:50.607785 11696 tablet_bootstrap.cc:492] T 5b363fa7f3a8413c93fa538e608ee72d P 456a4efb580f440c82b49a9ab397863f: No bootstrap required, opened a new log
I20260808 23:26:50.607981 11696 ts_tablet_manager.cc:1397] T 5b363fa7f3a8413c93fa538e608ee72d P 456a4efb580f440c82b49a9ab397863f: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260808 23:26:50.608305 10966 catalog_manager.cc:4841] TS ee373c494f1e45a8b19b0c545a924200 (127.9.44.67:39141): tablet 0901be26007f4d708274370790c0f0dc (table test-table [id=239b2362dbad4e8eb6a2f428e8cf9d52]) successfully deleted
I20260808 23:26:50.609138 11700 tablet_bootstrap.cc:492] T 5b363fa7f3a8413c93fa538e608ee72d P ee373c494f1e45a8b19b0c545a924200: No bootstrap required, opened a new log
I20260808 23:26:50.609404 11700 ts_tablet_manager.cc:1397] T 5b363fa7f3a8413c93fa538e608ee72d P ee373c494f1e45a8b19b0c545a924200: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.004s
I20260808 23:26:50.609354 11696 raft_consensus.cc:348] T 5b363fa7f3a8413c93fa538e608ee72d P 456a4efb580f440c82b49a9ab397863f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.609597 11696 raft_consensus.cc:374] T 5b363fa7f3a8413c93fa538e608ee72d P 456a4efb580f440c82b49a9ab397863f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:50.609717 11696 raft_consensus.cc:729] T 5b363fa7f3a8413c93fa538e608ee72d P 456a4efb580f440c82b49a9ab397863f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 456a4efb580f440c82b49a9ab397863f, State: Initialized, Role: FOLLOWER
I20260808 23:26:50.610044 11695 tablet_bootstrap.cc:654] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:50.610138 11734 ts_tablet_manager.cc:1905] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:26:50.610162 11700 raft_consensus.cc:348] T 5b363fa7f3a8413c93fa538e608ee72d P ee373c494f1e45a8b19b0c545a924200 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.610389 11700 raft_consensus.cc:374] T 5b363fa7f3a8413c93fa538e608ee72d P ee373c494f1e45a8b19b0c545a924200 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:50.610498 11700 raft_consensus.cc:729] T 5b363fa7f3a8413c93fa538e608ee72d P ee373c494f1e45a8b19b0c545a924200 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee373c494f1e45a8b19b0c545a924200, State: Initialized, Role: FOLLOWER
I20260808 23:26:50.610780 11700 consensus_queue.cc:260] T 5b363fa7f3a8413c93fa538e608ee72d P ee373c494f1e45a8b19b0c545a924200 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.610869 11696 consensus_queue.cc:260] T 5b363fa7f3a8413c93fa538e608ee72d P 456a4efb580f440c82b49a9ab397863f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.611415 11696 ts_tablet_manager.cc:1428] T 5b363fa7f3a8413c93fa538e608ee72d P 456a4efb580f440c82b49a9ab397863f: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 23:26:50.612838 11700 ts_tablet_manager.cc:1428] T 5b363fa7f3a8413c93fa538e608ee72d P ee373c494f1e45a8b19b0c545a924200: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260808 23:26:50.613750 11733 ts_tablet_manager.cc:1918] T 0901be26007f4d708274370790c0f0dc P 456a4efb580f440c82b49a9ab397863f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:26:50.613963 11733 log.cc:1196] T 0901be26007f4d708274370790c0f0dc P 456a4efb580f440c82b49a9ab397863f: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-0/wal/wals/0901be26007f4d708274370790c0f0dc
I20260808 23:26:50.614566 11733 ts_tablet_manager.cc:1939] T 0901be26007f4d708274370790c0f0dc P 456a4efb580f440c82b49a9ab397863f: Deleting consensus metadata
I20260808 23:26:50.616189 10968 catalog_manager.cc:4841] TS 456a4efb580f440c82b49a9ab397863f (127.9.44.65:45443): tablet 0901be26007f4d708274370790c0f0dc (table test-table [id=239b2362dbad4e8eb6a2f428e8cf9d52]) successfully deleted
I20260808 23:26:50.619645 11734 ts_tablet_manager.cc:1918] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:26:50.619843 11734 log.cc:1196] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/0901be26007f4d708274370790c0f0dc
I20260808 23:26:50.620404 11734 ts_tablet_manager.cc:1939] T 0901be26007f4d708274370790c0f0dc P b24abb45c55d4160b24f641734a5e793: Deleting consensus metadata
I20260808 23:26:50.622180 10969 catalog_manager.cc:4841] TS b24abb45c55d4160b24f641734a5e793 (127.9.44.66:42971): tablet 0901be26007f4d708274370790c0f0dc (table test-table [id=239b2362dbad4e8eb6a2f428e8cf9d52]) successfully deleted
I20260808 23:26:50.624045 11695 tablet_bootstrap.cc:492] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793: No bootstrap required, opened a new log
I20260808 23:26:50.624240 11695 ts_tablet_manager.cc:1397] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793: Time spent bootstrapping tablet: real 0.021s user 0.011s sys 0.000s
I20260808 23:26:50.625108 11695 raft_consensus.cc:348] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.625543 11695 raft_consensus.cc:374] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:50.625859 11695 raft_consensus.cc:729] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24abb45c55d4160b24f641734a5e793, State: Initialized, Role: FOLLOWER
I20260808 23:26:50.626252 11695 consensus_queue.cc:260] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.626917 11695 ts_tablet_manager.cc:1428] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 23:26:50.742813 11715 raft_consensus.cc:482] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:26:50.742997 11715 raft_consensus.cc:504] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.743647 11715 leader_election.cc:288] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 456a4efb580f440c82b49a9ab397863f (127.9.44.65:45443), ee373c494f1e45a8b19b0c545a924200 (127.9.44.67:39141)
I20260808 23:26:50.744289 11207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b363fa7f3a8413c93fa538e608ee72d" candidate_uuid: "b24abb45c55d4160b24f641734a5e793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "456a4efb580f440c82b49a9ab397863f" is_pre_election: true
I20260808 23:26:50.744341 11598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b363fa7f3a8413c93fa538e608ee72d" candidate_uuid: "b24abb45c55d4160b24f641734a5e793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee373c494f1e45a8b19b0c545a924200" is_pre_election: true
I20260808 23:26:50.744575 11207 raft_consensus.cc:2454] T 5b363fa7f3a8413c93fa538e608ee72d P 456a4efb580f440c82b49a9ab397863f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b24abb45c55d4160b24f641734a5e793 in term 0.
I20260808 23:26:50.744638 11598 raft_consensus.cc:2454] T 5b363fa7f3a8413c93fa538e608ee72d P ee373c494f1e45a8b19b0c545a924200 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b24abb45c55d4160b24f641734a5e793 in term 0.
I20260808 23:26:50.745095 11312 leader_election.cc:302] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 456a4efb580f440c82b49a9ab397863f, b24abb45c55d4160b24f641734a5e793; no voters:
I20260808 23:26:50.745385 11715 raft_consensus.cc:2781] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:26:50.745523 11715 raft_consensus.cc:482] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:26:50.745656 11715 raft_consensus.cc:3037] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:50.749847 11715 raft_consensus.cc:504] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.750748 11715 leader_election.cc:288] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793 [CANDIDATE]: Term 1 election: Requested vote from peers 456a4efb580f440c82b49a9ab397863f (127.9.44.65:45443), ee373c494f1e45a8b19b0c545a924200 (127.9.44.67:39141)
I20260808 23:26:50.751281 11207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b363fa7f3a8413c93fa538e608ee72d" candidate_uuid: "b24abb45c55d4160b24f641734a5e793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "456a4efb580f440c82b49a9ab397863f"
I20260808 23:26:50.751345 11598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b363fa7f3a8413c93fa538e608ee72d" candidate_uuid: "b24abb45c55d4160b24f641734a5e793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee373c494f1e45a8b19b0c545a924200"
I20260808 23:26:50.751502 11207 raft_consensus.cc:3037] T 5b363fa7f3a8413c93fa538e608ee72d P 456a4efb580f440c82b49a9ab397863f [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:50.751657 11598 raft_consensus.cc:3037] T 5b363fa7f3a8413c93fa538e608ee72d P ee373c494f1e45a8b19b0c545a924200 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:50.755214 11598 raft_consensus.cc:2454] T 5b363fa7f3a8413c93fa538e608ee72d P ee373c494f1e45a8b19b0c545a924200 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24abb45c55d4160b24f641734a5e793 in term 1.
I20260808 23:26:50.755266 11207 raft_consensus.cc:2454] T 5b363fa7f3a8413c93fa538e608ee72d P 456a4efb580f440c82b49a9ab397863f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24abb45c55d4160b24f641734a5e793 in term 1.
I20260808 23:26:50.755658 11310 leader_election.cc:302] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24abb45c55d4160b24f641734a5e793, ee373c494f1e45a8b19b0c545a924200; no voters:
I20260808 23:26:50.756140 11715 raft_consensus.cc:2781] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:26:50.757349 11715 raft_consensus.cc:686] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793 [term 1 LEADER]: Becoming Leader. State: Replica: b24abb45c55d4160b24f641734a5e793, State: Running, Role: LEADER
I20260808 23:26:50.757891 11715 consensus_queue.cc:237] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.764386 10985 catalog_manager.cc:5495] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793 reported cstate change: term changed from 0 to 1, leader changed from <none> to b24abb45c55d4160b24f641734a5e793 (127.9.44.66). New cstate: current_term: 1 leader_uuid: "b24abb45c55d4160b24f641734a5e793" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } health_report { overall_health: UNKNOWN } } }
Aug 08 23:26:50 dist-test-slave-6bbx krb5kdc[10927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231608, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:26:50.843796 11750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:50.822839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:40739 (local address 127.0.0.1:38566)
0808 23:26:50.826022 (+ 3183us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:50.826044 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:50.826175 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:50.826541 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:50.826544 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:50.827474 (+ 930us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:26:50.827837 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:50.827848 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.829583 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.829587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:50.830815 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:50.830829 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.831900 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.831906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:50.832081 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:50.832851 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:26:50.832897 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:26:50.835820 (+ 2923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:26:50.840737 (+ 4917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:50.840753 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:50.840760 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:50.841365 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:50.841857 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:50.841860 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:50.841862 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:50.842077 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:50.842700 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:26:50.842712 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:26:50.842918 (+ 206us) client_negotiation.cc:769] Sending connection context
0808 23:26:50.843073 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 23:26:50.843266 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3008,"thread_start_us":2866,"threads_started":1}
I20260808 23:26:50.844187 11749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:50.822559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38566 (local address 127.9.44.126:40739)
0808 23:26:50.824224 (+ 1665us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:50.824231 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:50.826145 (+ 1914us) server_negotiation.cc:407] Connection header received
0808 23:26:50.826307 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:50.826313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:50.826391 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:50.826509 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:50.828018 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.829447 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.830959 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:50.831598 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:50.831770 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:50.836031 (+ 4261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:50.836039 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:50.836044 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:50.836103 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:50.840580 (+ 4477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:50.841494 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:50.841500 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:50.841505 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:50.841751 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:50.842195 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:50.842200 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:50.842203 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:50.842577 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:50.842776 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:50.843263 (+ 487us) server_negotiation.cc:299] Negotiation successful
0808 23:26:50.843476 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1503,"thread_start_us":1319,"threads_started":1}
I20260808 23:26:50.849342 10985 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:38566:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 23:26:50.850637 10985 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:26:50.857731 11374 tablet_service.cc:1467] Processing CreateTablet for tablet 9e8f25a623dd4021854af75cd20470a8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b08945afe99944d89d6cf7d282d59ba1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 23:26:50.858302 11374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e8f25a623dd4021854af75cd20470a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:50.859961 11182 tablet_service.cc:1467] Processing CreateTablet for tablet 9e8f25a623dd4021854af75cd20470a8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b08945afe99944d89d6cf7d282d59ba1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 23:26:50.860440 11182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e8f25a623dd4021854af75cd20470a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:50.866381 11563 tablet_service.cc:1467] Processing CreateTablet for tablet 9e8f25a623dd4021854af75cd20470a8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b08945afe99944d89d6cf7d282d59ba1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 23:26:50.867036 11696 tablet_bootstrap.cc:492] T 9e8f25a623dd4021854af75cd20470a8 P 456a4efb580f440c82b49a9ab397863f: Bootstrap starting.
I20260808 23:26:50.867149 11563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e8f25a623dd4021854af75cd20470a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:50.869980 11696 tablet_bootstrap.cc:654] T 9e8f25a623dd4021854af75cd20470a8 P 456a4efb580f440c82b49a9ab397863f: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:50.871244 11695 tablet_bootstrap.cc:492] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793: Bootstrap starting.
I20260808 23:26:50.873778 11695 tablet_bootstrap.cc:654] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:50.876852 11700 tablet_bootstrap.cc:492] T 9e8f25a623dd4021854af75cd20470a8 P ee373c494f1e45a8b19b0c545a924200: Bootstrap starting.
I20260808 23:26:50.878549 11696 tablet_bootstrap.cc:492] T 9e8f25a623dd4021854af75cd20470a8 P 456a4efb580f440c82b49a9ab397863f: No bootstrap required, opened a new log
I20260808 23:26:50.878787 11696 ts_tablet_manager.cc:1397] T 9e8f25a623dd4021854af75cd20470a8 P 456a4efb580f440c82b49a9ab397863f: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.007s
I20260808 23:26:50.879370 11700 tablet_bootstrap.cc:654] T 9e8f25a623dd4021854af75cd20470a8 P ee373c494f1e45a8b19b0c545a924200: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:50.879609 11696 raft_consensus.cc:348] T 9e8f25a623dd4021854af75cd20470a8 P 456a4efb580f440c82b49a9ab397863f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.879927 11696 raft_consensus.cc:374] T 9e8f25a623dd4021854af75cd20470a8 P 456a4efb580f440c82b49a9ab397863f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:50.880023 11696 raft_consensus.cc:729] T 9e8f25a623dd4021854af75cd20470a8 P 456a4efb580f440c82b49a9ab397863f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 456a4efb580f440c82b49a9ab397863f, State: Initialized, Role: FOLLOWER
I20260808 23:26:50.880463 11696 consensus_queue.cc:260] T 9e8f25a623dd4021854af75cd20470a8 P 456a4efb580f440c82b49a9ab397863f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.881075 11709 tablet_replica.cc:439] T 9e8f25a623dd4021854af75cd20470a8 P 456a4efb580f440c82b49a9ab397863f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } } }
I20260808 23:26:50.881495 11709 tablet_replica.cc:442] T 9e8f25a623dd4021854af75cd20470a8 P 456a4efb580f440c82b49a9ab397863f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:26:50.882565 11696 ts_tablet_manager.cc:1428] T 9e8f25a623dd4021854af75cd20470a8 P 456a4efb580f440c82b49a9ab397863f: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 23:26:50.884560 11695 tablet_bootstrap.cc:492] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793: No bootstrap required, opened a new log
I20260808 23:26:50.884757 11695 ts_tablet_manager.cc:1397] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20260808 23:26:50.885812 11695 raft_consensus.cc:348] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.886063 11695 raft_consensus.cc:374] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:50.886161 11695 raft_consensus.cc:729] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24abb45c55d4160b24f641734a5e793, State: Initialized, Role: FOLLOWER
I20260808 23:26:50.886552 11700 tablet_bootstrap.cc:492] T 9e8f25a623dd4021854af75cd20470a8 P ee373c494f1e45a8b19b0c545a924200: No bootstrap required, opened a new log
I20260808 23:26:50.886459 11695 consensus_queue.cc:260] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.886827 11700 ts_tablet_manager.cc:1397] T 9e8f25a623dd4021854af75cd20470a8 P ee373c494f1e45a8b19b0c545a924200: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260808 23:26:50.887012 11715 tablet_replica.cc:439] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } } }
I20260808 23:26:50.887308 11715 tablet_replica.cc:442] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:26:50.887409 11695 ts_tablet_manager.cc:1428] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:26:50.887854 11700 raft_consensus.cc:348] T 9e8f25a623dd4021854af75cd20470a8 P ee373c494f1e45a8b19b0c545a924200 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.888173 11700 raft_consensus.cc:374] T 9e8f25a623dd4021854af75cd20470a8 P ee373c494f1e45a8b19b0c545a924200 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:50.888345 11700 raft_consensus.cc:729] T 9e8f25a623dd4021854af75cd20470a8 P ee373c494f1e45a8b19b0c545a924200 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee373c494f1e45a8b19b0c545a924200, State: Initialized, Role: FOLLOWER
I20260808 23:26:50.888695 11700 consensus_queue.cc:260] T 9e8f25a623dd4021854af75cd20470a8 P ee373c494f1e45a8b19b0c545a924200 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.889423 11700 ts_tablet_manager.cc:1428] T 9e8f25a623dd4021854af75cd20470a8 P ee373c494f1e45a8b19b0c545a924200: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 23:26:50.889292 11711 tablet_replica.cc:439] T 9e8f25a623dd4021854af75cd20470a8 P ee373c494f1e45a8b19b0c545a924200: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } } }
I20260808 23:26:50.889815 11711 tablet_replica.cc:442] T 9e8f25a623dd4021854af75cd20470a8 P ee373c494f1e45a8b19b0c545a924200: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:26:50.969686 11715 raft_consensus.cc:482] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:26:50.969901 11715 raft_consensus.cc:504] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.970537 11715 leader_election.cc:288] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 456a4efb580f440c82b49a9ab397863f (127.9.44.65:45443), ee373c494f1e45a8b19b0c545a924200 (127.9.44.67:39141)
I20260808 23:26:50.971086 11207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e8f25a623dd4021854af75cd20470a8" candidate_uuid: "b24abb45c55d4160b24f641734a5e793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "456a4efb580f440c82b49a9ab397863f" is_pre_election: true
I20260808 23:26:50.971220 11598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e8f25a623dd4021854af75cd20470a8" candidate_uuid: "b24abb45c55d4160b24f641734a5e793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee373c494f1e45a8b19b0c545a924200" is_pre_election: true
I20260808 23:26:50.971401 11207 raft_consensus.cc:2454] T 9e8f25a623dd4021854af75cd20470a8 P 456a4efb580f440c82b49a9ab397863f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b24abb45c55d4160b24f641734a5e793 in term 0.
I20260808 23:26:50.971493 11598 raft_consensus.cc:2454] T 9e8f25a623dd4021854af75cd20470a8 P ee373c494f1e45a8b19b0c545a924200 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b24abb45c55d4160b24f641734a5e793 in term 0.
I20260808 23:26:50.971968 11310 leader_election.cc:302] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24abb45c55d4160b24f641734a5e793, ee373c494f1e45a8b19b0c545a924200; no voters:
I20260808 23:26:50.972350 11715 raft_consensus.cc:2781] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:26:50.972438 11715 raft_consensus.cc:482] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:26:50.972604 11715 raft_consensus.cc:3037] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:50.976485 11715 raft_consensus.cc:504] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.977263 11715 leader_election.cc:288] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793 [CANDIDATE]: Term 1 election: Requested vote from peers 456a4efb580f440c82b49a9ab397863f (127.9.44.65:45443), ee373c494f1e45a8b19b0c545a924200 (127.9.44.67:39141)
I20260808 23:26:50.977571 11207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e8f25a623dd4021854af75cd20470a8" candidate_uuid: "b24abb45c55d4160b24f641734a5e793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "456a4efb580f440c82b49a9ab397863f"
I20260808 23:26:50.977653 11598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e8f25a623dd4021854af75cd20470a8" candidate_uuid: "b24abb45c55d4160b24f641734a5e793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee373c494f1e45a8b19b0c545a924200"
I20260808 23:26:50.977849 11598 raft_consensus.cc:3037] T 9e8f25a623dd4021854af75cd20470a8 P ee373c494f1e45a8b19b0c545a924200 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:50.977835 11207 raft_consensus.cc:3037] T 9e8f25a623dd4021854af75cd20470a8 P 456a4efb580f440c82b49a9ab397863f [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:50.979935 11598 raft_consensus.cc:2454] T 9e8f25a623dd4021854af75cd20470a8 P ee373c494f1e45a8b19b0c545a924200 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24abb45c55d4160b24f641734a5e793 in term 1.
I20260808 23:26:50.980470 11310 leader_election.cc:302] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b24abb45c55d4160b24f641734a5e793, ee373c494f1e45a8b19b0c545a924200; no voters:
I20260808 23:26:50.980697 11715 raft_consensus.cc:2781] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:26:50.981040 11207 raft_consensus.cc:2454] T 9e8f25a623dd4021854af75cd20470a8 P 456a4efb580f440c82b49a9ab397863f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b24abb45c55d4160b24f641734a5e793 in term 1.
I20260808 23:26:50.980876 11715 raft_consensus.cc:686] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793 [term 1 LEADER]: Becoming Leader. State: Replica: b24abb45c55d4160b24f641734a5e793, State: Running, Role: LEADER
I20260808 23:26:50.981380 11715 consensus_queue.cc:237] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } }
I20260808 23:26:50.983146 11708 tablet_replica.cc:439] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793: TxnStatusTablet state changed. Reason: New leader b24abb45c55d4160b24f641734a5e793. Latest consensus state: current_term: 1 leader_uuid: "b24abb45c55d4160b24f641734a5e793" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } } }
I20260808 23:26:50.983656 11708 tablet_replica.cc:442] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793: This TxnStatusTablet replica's current role is: LEADER
I20260808 23:26:50.984344 11756 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 23:26:50.984632 11756 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 23:26:50.984743 10984 catalog_manager.cc:5495] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793 reported cstate change: term changed from 0 to 1, leader changed from <none> to b24abb45c55d4160b24f641734a5e793 (127.9.44.66). New cstate: current_term: 1 leader_uuid: "b24abb45c55d4160b24f641734a5e793" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } health_report { overall_health: UNKNOWN } } }
I20260808 23:26:51.008308 11750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:51.001979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:42971 (local address 127.0.0.1:57306)
0808 23:26:51.002219 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:51.002237 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:51.002342 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:51.002725 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:51.002728 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:51.002738 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 23:26:51.002950 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:51.002960 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:51.004681 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:51.004686 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:51.006135 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:51.006151 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:51.007194 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:51.007200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:51.007354 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:51.007412 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 23:26:51.007895 (+ 483us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 23:26:51.007914 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 23:26:51.007972 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 23:26:51.008083 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 23:26:51.008446 11687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:51.002090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57306 (local address 127.9.44.66:42971)
0808 23:26:51.002281 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:51.002286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:51.002307 (+ 21us) server_negotiation.cc:407] Connection header received
0808 23:26:51.002510 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:51.002516 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:51.002609 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:51.002715 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:26:51.003087 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:51.004537 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:51.006284 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:51.007021 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:51.007169 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:51.007591 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:26:51.007783 (+ 192us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:26:51.007887 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:51.008028 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 23:26:51.008108 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 23:26:51.012455 11598 raft_consensus.cc:1261] T 9e8f25a623dd4021854af75cd20470a8 P ee373c494f1e45a8b19b0c545a924200 [term 1 FOLLOWER]: Refusing update from remote peer b24abb45c55d4160b24f641734a5e793: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:26:51.012669 11207 raft_consensus.cc:1261] T 9e8f25a623dd4021854af75cd20470a8 P 456a4efb580f440c82b49a9ab397863f [term 1 FOLLOWER]: Refusing update from remote peer b24abb45c55d4160b24f641734a5e793: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:26:51.013116 11708 consensus_queue.cc:1035] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:26:51.013455 11715 consensus_queue.cc:1035] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793 [LEADER]: Connected to new peer: Peer: permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:26:51.016610 11711 tablet_replica.cc:439] T 9e8f25a623dd4021854af75cd20470a8 P ee373c494f1e45a8b19b0c545a924200: TxnStatusTablet state changed. Reason: New leader b24abb45c55d4160b24f641734a5e793. Latest consensus state: current_term: 1 leader_uuid: "b24abb45c55d4160b24f641734a5e793" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } } }
I20260808 23:26:51.016860 11711 tablet_replica.cc:442] T 9e8f25a623dd4021854af75cd20470a8 P ee373c494f1e45a8b19b0c545a924200: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:26:51.017928 11709 tablet_replica.cc:439] T 9e8f25a623dd4021854af75cd20470a8 P 456a4efb580f440c82b49a9ab397863f: TxnStatusTablet state changed. Reason: New leader b24abb45c55d4160b24f641734a5e793. Latest consensus state: current_term: 1 leader_uuid: "b24abb45c55d4160b24f641734a5e793" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } } }
I20260808 23:26:51.018193 11709 tablet_replica.cc:442] T 9e8f25a623dd4021854af75cd20470a8 P 456a4efb580f440c82b49a9ab397863f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:26:51.019650 11729 tablet_replica.cc:439] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b24abb45c55d4160b24f641734a5e793" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } } }
I20260808 23:26:51.020361 11729 tablet_replica.cc:442] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793: This TxnStatusTablet replica's current role is: LEADER
I20260808 23:26:51.020191 11715 tablet_replica.cc:439] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b24abb45c55d4160b24f641734a5e793" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } } }
I20260808 23:26:51.020539 11715 tablet_replica.cc:442] T 9e8f25a623dd4021854af75cd20470a8 P b24abb45c55d4160b24f641734a5e793: This TxnStatusTablet replica's current role is: LEADER
I20260808 23:26:51.021023 11709 tablet_replica.cc:439] T 9e8f25a623dd4021854af75cd20470a8 P 456a4efb580f440c82b49a9ab397863f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b24abb45c55d4160b24f641734a5e793" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } } }
I20260808 23:26:51.021318 11709 tablet_replica.cc:442] T 9e8f25a623dd4021854af75cd20470a8 P 456a4efb580f440c82b49a9ab397863f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:26:51.025341 11719 mvcc.cc:205] Tried to move back new op lower bound from 7316404678701158400 to 7316404678581911552. Current Snapshot: MvccSnapshot[applied={T|T < 7316404678701158400 or (T in {7316404678701158400})}]
I20260808 23:26:51.025445 11711 tablet_replica.cc:439] T 9e8f25a623dd4021854af75cd20470a8 P ee373c494f1e45a8b19b0c545a924200: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b24abb45c55d4160b24f641734a5e793" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 } } peers { permanent_uuid: "b24abb45c55d4160b24f641734a5e793" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 42971 } } peers { permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 } } }
I20260808 23:26:51.025661 11711 tablet_replica.cc:442] T 9e8f25a623dd4021854af75cd20470a8 P ee373c494f1e45a8b19b0c545a924200: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:26:51.051409 11687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:51.041192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:55205 (local address 127.9.44.66:42971)
0808 23:26:51.041392 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:51.041401 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:51.044485 (+ 3084us) server_negotiation.cc:407] Connection header received
0808 23:26:51.044613 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:51.044618 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:51.044720 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:51.044829 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:26:51.046065 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:51.047221 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:51.049224 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:51.049946 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:51.050060 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:51.050592 (+ 532us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:26:51.050690 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:26:51.050762 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:51.050953 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 23:26:51.051015 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 23:26:51.051692 11765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:51.041056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:42971 (local address 127.9.44.66:55205)
0808 23:26:51.044372 (+ 3316us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:51.044387 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:51.044499 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:51.044843 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:51.044846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:51.044856 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 23:26:51.045791 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:51.045804 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:51.047341 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:51.047345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:51.049086 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:51.049098 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:51.050247 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:51.050254 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:51.050379 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:51.050434 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 23:26:51.050792 (+ 358us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 23:26:51.050801 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 23:26:51.050865 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 23:26:51.051004 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3157,"thread_start_us":2947,"threads_started":1}
I20260808 23:26:51.061589 11207 raft_consensus.cc:1261] T 5b363fa7f3a8413c93fa538e608ee72d P 456a4efb580f440c82b49a9ab397863f [term 1 FOLLOWER]: Refusing update from remote peer b24abb45c55d4160b24f641734a5e793: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:26:51.061643 11598 raft_consensus.cc:1261] T 5b363fa7f3a8413c93fa538e608ee72d P ee373c494f1e45a8b19b0c545a924200 [term 1 FOLLOWER]: Refusing update from remote peer b24abb45c55d4160b24f641734a5e793: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:26:51.062270 11708 consensus_queue.cc:1035] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793 [LEADER]: Connected to new peer: Peer: permanent_uuid: "456a4efb580f440c82b49a9ab397863f" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:26:51.062549 11715 consensus_queue.cc:1035] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee373c494f1e45a8b19b0c545a924200" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:26:51.073645 11723 mvcc.cc:205] Tried to move back new op lower bound from 7316404678903463936 to 7316404677667475456. Current Snapshot: MvccSnapshot[applied={T|T < 7316404678903463936}]
I20260808 23:26:51.132643 10985 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:26:51.132882 10985 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:26:51.135900 10985 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 723cab76a5c14d61bb8fffc84a3ea699: Sending DeleteTablet for 3 replicas of tablet 5b363fa7f3a8413c93fa538e608ee72d
I20260808 23:26:51.136596 11182 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b363fa7f3a8413c93fa538e608ee72d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:26:51 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:46526
I20260808 23:26:51.136861 11563 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b363fa7f3a8413c93fa538e608ee72d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:26:51 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:41732
I20260808 23:26:51.137286 11374 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b363fa7f3a8413c93fa538e608ee72d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:26:51 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:57284
I20260808 23:26:51.145012 11772 tablet_replica.cc:330] T 5b363fa7f3a8413c93fa538e608ee72d P ee373c494f1e45a8b19b0c545a924200: stopping tablet replica
I20260808 23:26:51.145313 11772 raft_consensus.cc:2229] T 5b363fa7f3a8413c93fa538e608ee72d P ee373c494f1e45a8b19b0c545a924200 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:26:51.145686 11772 raft_consensus.cc:2258] T 5b363fa7f3a8413c93fa538e608ee72d P ee373c494f1e45a8b19b0c545a924200 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:26:51.148347 11774 tablet_replica.cc:330] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793: stopping tablet replica
I20260808 23:26:51.148542 11772 ts_tablet_manager.cc:1905] T 5b363fa7f3a8413c93fa538e608ee72d P ee373c494f1e45a8b19b0c545a924200: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:26:51.148650 11774 raft_consensus.cc:2229] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:26:51.149096 11774 raft_consensus.cc:2258] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:26:51.150307 11774 ts_tablet_manager.cc:1905] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:26:51.159355 11771 tablet_replica.cc:330] T 5b363fa7f3a8413c93fa538e608ee72d P 456a4efb580f440c82b49a9ab397863f: stopping tablet replica
I20260808 23:26:51.159683 11771 raft_consensus.cc:2229] T 5b363fa7f3a8413c93fa538e608ee72d P 456a4efb580f440c82b49a9ab397863f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:26:51.160054 11771 raft_consensus.cc:2258] T 5b363fa7f3a8413c93fa538e608ee72d P 456a4efb580f440c82b49a9ab397863f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:26:51.161240 11771 ts_tablet_manager.cc:1905] T 5b363fa7f3a8413c93fa538e608ee72d P 456a4efb580f440c82b49a9ab397863f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:26:51.163040 11772 ts_tablet_manager.cc:1918] T 5b363fa7f3a8413c93fa538e608ee72d P ee373c494f1e45a8b19b0c545a924200: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:26:51.163254 11772 log.cc:1196] T 5b363fa7f3a8413c93fa538e608ee72d P ee373c494f1e45a8b19b0c545a924200: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/5b363fa7f3a8413c93fa538e608ee72d
I20260808 23:26:51.163810 11772 ts_tablet_manager.cc:1939] T 5b363fa7f3a8413c93fa538e608ee72d P ee373c494f1e45a8b19b0c545a924200: Deleting consensus metadata
I20260808 23:26:51.164822 10966 catalog_manager.cc:4841] TS ee373c494f1e45a8b19b0c545a924200 (127.9.44.67:39141): tablet 5b363fa7f3a8413c93fa538e608ee72d (table test-table [id=33b9d7e3abaf465d813b9ec83c6621f9]) successfully deleted
I20260808 23:26:51.163812 11774 ts_tablet_manager.cc:1918] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:26:51.165170 11774 log.cc:1196] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/5b363fa7f3a8413c93fa538e608ee72d
I20260808 23:26:51.165877 11774 ts_tablet_manager.cc:1939] T 5b363fa7f3a8413c93fa538e608ee72d P b24abb45c55d4160b24f641734a5e793: Deleting consensus metadata
I20260808 23:26:51.167026 10969 catalog_manager.cc:4841] TS b24abb45c55d4160b24f641734a5e793 (127.9.44.66:42971): tablet 5b363fa7f3a8413c93fa538e608ee72d (table test-table [id=33b9d7e3abaf465d813b9ec83c6621f9]) successfully deleted
I20260808 23:26:51.168080 11771 ts_tablet_manager.cc:1918] T 5b363fa7f3a8413c93fa538e608ee72d P 456a4efb580f440c82b49a9ab397863f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:26:51.168258 11771 log.cc:1196] T 5b363fa7f3a8413c93fa538e608ee72d P 456a4efb580f440c82b49a9ab397863f: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786231597224934-9393-0/minicluster-data/ts-0/wal/wals/5b363fa7f3a8413c93fa538e608ee72d
I20260808 23:26:51.168870 11771 ts_tablet_manager.cc:1939] T 5b363fa7f3a8413c93fa538e608ee72d P 456a4efb580f440c82b49a9ab397863f: Deleting consensus metadata
I20260808 23:26:51.169823 10968 catalog_manager.cc:4841] TS 456a4efb580f440c82b49a9ab397863f (127.9.44.65:45443): tablet 5b363fa7f3a8413c93fa538e608ee72d (table test-table [id=33b9d7e3abaf465d813b9ec83c6621f9]) successfully deleted
Aug 08 23:26:51 dist-test-slave-6bbx krb5kdc[10927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231610, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.44.65@KRBTEST.COM
I20260808 23:26:51.183252 11776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:51.152546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46536 (local address 127.9.44.65:45443)
0808 23:26:51.163947 (+ 11401us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:51.163957 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:51.163976 (+ 19us) server_negotiation.cc:407] Connection header received
0808 23:26:51.164033 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:51.164038 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:51.164117 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:51.167106 (+ 2989us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:51.167255 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:51.168578 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:51.170470 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:51.171225 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:51.171357 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:51.175984 (+ 4627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:51.176002 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:51.176021 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:51.176059 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:51.180031 (+ 3972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:51.181012 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:51.181021 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:51.181025 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:51.181231 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:51.181592 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:51.181597 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:51.181600 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:51.182095 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:51.182391 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:51.182634 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 23:26:51.182867 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11171,"thread_start_us":11677,"threads_started":1}
W20260808 23:26:51.184124 11274 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46536
I20260808 23:26:51.202584 11749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:51.187116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38578 (local address 127.9.44.126:40739)
0808 23:26:51.187421 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:51.187428 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:51.187559 (+ 131us) server_negotiation.cc:407] Connection header received
0808 23:26:51.187722 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:51.187727 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:51.187846 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:51.187936 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:51.189531 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:51.190793 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:51.192420 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:51.193155 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:51.193275 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:51.195529 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:51.195536 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:51.195540 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:51.195593 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:51.199422 (+ 3829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:51.200047 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:51.200052 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:51.200056 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:51.200233 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:51.200567 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:51.200572 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:51.200576 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:51.201065 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:51.201324 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:51.201773 (+ 449us) server_negotiation.cc:299] Negotiation successful
0808 23:26:51.202078 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182}
W20260808 23:26:51.203284 10985 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38578
I20260808 23:26:51.205473 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 11090
I20260808 23:26:51.226522 11776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:51.206260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46538 (local address 127.9.44.65:45443)
0808 23:26:51.206437 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:51.206443 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:51.211161 (+ 4718us) server_negotiation.cc:407] Connection header received
0808 23:26:51.211324 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:51.211329 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:51.211410 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:51.211515 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:51.212829 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:51.213984 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:51.215235 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:51.215975 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:51.216121 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:51.218489 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:51.218496 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:51.218500 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:51.218541 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:51.223313 (+ 4772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:51.224117 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:51.224123 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:51.224127 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:51.224355 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:51.224781 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:51.224786 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:51.224789 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:51.225220 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:51.225518 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:51.225773 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 23:26:51.226056 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260808 23:26:51.227171 11274 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46538
W20260808 23:26:51.227806 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 11090: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:26:51.228056 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 11090
I20260808 23:26:51.262861 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 11286
Aug 08 23:26:51 dist-test-slave-6bbx krb5kdc[10927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231610, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.44.66@KRBTEST.COM
I20260808 23:26:51.281347 11687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:51.263613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57318 (local address 127.9.44.66:42971)
0808 23:26:51.263874 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:51.263880 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:51.263903 (+ 23us) server_negotiation.cc:407] Connection header received
0808 23:26:51.264141 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:51.264147 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:51.264239 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:51.264347 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:51.265969 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:51.267085 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:51.268922 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:51.269654 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:51.269788 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:51.274819 (+ 5031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:51.274841 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:51.274861 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:51.274902 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:51.278559 (+ 3657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:51.279196 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:51.279205 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:51.279209 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:51.279361 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:51.279746 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:51.279752 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:51.279756 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:51.280218 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:51.280519 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:51.280731 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 23:26:51.280950 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
W20260808 23:26:51.281992 11468 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57318
W20260808 23:26:51.282588 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 11286: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:26:51.282742 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 11286
I20260808 23:26:51.310318 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 11481
Aug 08 23:26:51 dist-test-slave-6bbx krb5kdc[10927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231610, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.44.67@KRBTEST.COM
I20260808 23:26:51.324057 11782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:51.311053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41740 (local address 127.9.44.67:39141)
0808 23:26:51.311576 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:51.311582 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:51.311602 (+ 20us) server_negotiation.cc:407] Connection header received
0808 23:26:51.311680 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:51.311685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:51.311754 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:51.311848 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:51.312999 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:51.313754 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:51.314888 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:51.315317 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:51.315405 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:51.319165 (+ 3760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:51.319177 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:51.319192 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:51.319224 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:51.321888 (+ 2664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:51.322418 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:51.322424 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:51.322426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:51.322562 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:51.322843 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:51.322846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:51.322848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:51.323154 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:51.323391 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:51.323532 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 23:26:51.323737 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":267,"threads_started":1}
W20260808 23:26:51.324703 11659 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41740
W20260808 23:26:51.325294 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 11481: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:26:51.325420 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 11481
I20260808 23:26:51.344131 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 10943
I20260808 23:26:51.345109 11043 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:26:51.457021 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 10943
2026-08-08T23:26:51Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4519 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:26:51 dist-test-slave-6bbx krb5kdc[11786](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:26:51 dist-test-slave-6bbx krb5kdc[11786](info): set up 2 sockets
Aug 08 23:26:51 dist-test-slave-6bbx krb5kdc[11786](info): commencing operation
krb5kdc: starting...
W20260808 23:26:52.626212 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:26:52 dist-test-slave-6bbx krb5kdc[11786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:26:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:26:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:26:52.800591 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:44905
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:34267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:44905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:26:53.111696 11808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:26:53.111974 11808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:26:53.112105 11808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:26:53.118892 11808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:26:53.118980 11808 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 23:26:53.119048 11808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:26:53.119117 11808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:26:53.119179 11808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:26:53.119266 11808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:26:53.128404 11808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:34267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:44905
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:44905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:26:53.129482 11808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:26:53.130692 11808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:26:53.141548 11815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:53.141577 11817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:53.141611 11814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:26:53.142382 11808 server_base.cc:1034] running on GCE node
I20260808 23:26:53.143146 11808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:26:53.144428 11808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:26:53.145641 11808 hybrid_clock.cc:648] HybridClock initialized: now 1786231613145576 us; error 36 us; skew 500 ppm
I20260808 23:26:53.145936 11808 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:26:53 dist-test-slave-6bbx krb5kdc[11786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231613, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:26:53.148886 11808 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:26:53.150139 11808 webserver.cc:460] Webserver started at http://127.9.44.126:45249/ using document root <none> and password file <none>
I20260808 23:26:53.150769 11808 fs_manager.cc:359] Metadata directory not provided
I20260808 23:26:53.150868 11808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:26:53.151157 11808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:26:53.154031 11808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "ed4ee6cf29134b0889de70142db69280"
format_stamp: "Formatted at 2026-08-08 23:26:53 on dist-test-slave-6bbx"
server_key: "458da63a6db8b710bea6908c55b66f0f"
server_key_iv: "bbeb503fa34424cf33427f61a13d0db9"
server_key_version: "encryptionkey@0"
I20260808 23:26:53.154764 11808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "ed4ee6cf29134b0889de70142db69280"
format_stamp: "Formatted at 2026-08-08 23:26:53 on dist-test-slave-6bbx"
server_key: "458da63a6db8b710bea6908c55b66f0f"
server_key_iv: "bbeb503fa34424cf33427f61a13d0db9"
server_key_version: "encryptionkey@0"
I20260808 23:26:53.159464 11808 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.007s
I20260808 23:26:53.162653 11826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:53.163894 11808 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:26:53.164039 11808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "ed4ee6cf29134b0889de70142db69280"
format_stamp: "Formatted at 2026-08-08 23:26:53 on dist-test-slave-6bbx"
server_key: "458da63a6db8b710bea6908c55b66f0f"
server_key_iv: "bbeb503fa34424cf33427f61a13d0db9"
server_key_version: "encryptionkey@0"
I20260808 23:26:53.164224 11808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:26:53.175527 11808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:26:53.178860 11808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:26:53.179085 11808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:26:53.197584 11808 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:44905
I20260808 23:26:53.197603 11926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:44905 every 8 connection(s)
I20260808 23:26:53.198756 11808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:26:53.202198 11927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:53.203510 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 11808
I20260808 23:26:53.203716 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:26:53.204329 9393 external_mini_cluster.cc:1428] Setting key 6fa78c1047929d3a948cbaa67f9c4525
I20260808 23:26:53.215075 11927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280: Bootstrap starting.
Aug 08 23:26:53 dist-test-slave-6bbx krb5kdc[11786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:26:53.218607 11927 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:53.219611 11927 log.cc:824] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280: Log is configured to *not* fsync() on all Append() calls
I20260808 23:26:53.222172 11927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280: No bootstrap required, opened a new log
I20260808 23:26:53.225672 11930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:53.206063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49576 (local address 127.9.44.126:44905)
0808 23:26:53.206795 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:53.206805 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:53.206865 (+ 60us) server_negotiation.cc:407] Connection header received
0808 23:26:53.208479 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:53.208500 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:53.208832 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:53.209122 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:53.210853 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:53.212152 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:53.213886 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:53.214509 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:53.214639 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:53.219080 (+ 4441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:53.219087 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:53.219100 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:53.219134 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:53.221924 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:53.222585 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:53.222591 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:53.222596 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:53.222758 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:53.223106 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:53.223110 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:53.223112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:53.223623 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:53.223942 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:53.224394 (+ 452us) server_negotiation.cc:299] Negotiation successful
0808 23:26:53.224761 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":332,"threads_started":1}
I20260808 23:26:53.229663 11927 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed4ee6cf29134b0889de70142db69280" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44905 } }
I20260808 23:26:53.229996 11927 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:53.230088 11927 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed4ee6cf29134b0889de70142db69280, State: Initialized, Role: FOLLOWER
I20260808 23:26:53.230623 11927 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed4ee6cf29134b0889de70142db69280" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44905 } }
I20260808 23:26:53.230840 11927 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:26:53.230969 11927 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:26:53.231118 11927 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:53.233398 11927 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed4ee6cf29134b0889de70142db69280" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44905 } }
I20260808 23:26:53.233848 11927 leader_election.cc:302] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed4ee6cf29134b0889de70142db69280; no voters:
I20260808 23:26:53.234367 11927 leader_election.cc:288] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:26:53.234474 11933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:26:53.235857 11933 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280 [term 1 LEADER]: Becoming Leader. State: Replica: ed4ee6cf29134b0889de70142db69280, State: Running, Role: LEADER
I20260808 23:26:53.236423 11927 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:26:53.236351 11933 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed4ee6cf29134b0889de70142db69280" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44905 } }
I20260808 23:26:53.238713 11935 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed4ee6cf29134b0889de70142db69280. Latest consensus state: current_term: 1 leader_uuid: "ed4ee6cf29134b0889de70142db69280" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed4ee6cf29134b0889de70142db69280" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44905 } } }
I20260808 23:26:53.238797 11934 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed4ee6cf29134b0889de70142db69280" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed4ee6cf29134b0889de70142db69280" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44905 } } }
I20260808 23:26:53.239042 11935 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280 [sys.catalog]: This master's current role is: LEADER
I20260808 23:26:53.239148 11934 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280 [sys.catalog]: This master's current role is: LEADER
I20260808 23:26:53.242352 11941 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:26:53.248153 11941 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:26:53.256726 11941 catalog_manager.cc:1347] Generated new cluster ID: 3df9020b690e48579f9111f53a0095e8
I20260808 23:26:53.256839 11941 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:26:53.263314 11941 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:26:53.265067 11941 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:26:53.276503 11941 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280: Generated new TSK 0
I20260808 23:26:53.277391 11941 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.44.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
I20260808 23:26:53.328102 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:0
--local_ip_for_outbound_sockets=127.9.44.65
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:44905
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:34267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:26:53.650619 11957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:26:53.650902 11957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:26:53.651044 11957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:26:53.658186 11957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:26:53.658301 11957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:26:53.658419 11957 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 23:26:53.658484 11957 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 23:26:53.658586 11957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:26:53.668675 11957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:34267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.44.126:44905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:26:53.669874 11957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:26:53.671159 11957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:26:53.683657 11968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:53.683739 11973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:53.683838 11966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:26:53.684379 11957 server_base.cc:1034] running on GCE node
I20260808 23:26:53.685009 11957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:26:53.685885 11957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:26:53.687089 11957 hybrid_clock.cc:648] HybridClock initialized: now 1786231613687031 us; error 50 us; skew 500 ppm
I20260808 23:26:53.687379 11957 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:26:53 dist-test-slave-6bbx krb5kdc[11786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231613, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:26:53.694536 11957 init.cc:377] Logged in from keytab as kudu/127.9.44.65@KRBTEST.COM (short username kudu)
I20260808 23:26:53.695889 11957 webserver.cc:460] Webserver started at http://127.9.44.65:40345/ using document root <none> and password file <none>
I20260808 23:26:53.696506 11957 fs_manager.cc:359] Metadata directory not provided
I20260808 23:26:53.696604 11957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:26:53.696871 11957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:26:53.699568 11957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-0/data/instance:
uuid: "1fe6b02094dd4d8aa91c77bf1a44299e"
format_stamp: "Formatted at 2026-08-08 23:26:53 on dist-test-slave-6bbx"
server_key: "ed9b3dac69d7cb4223ac9daefb8f5d02"
server_key_iv: "1ab2cb9392e0c5aade96c70524999abb"
server_key_version: "encryptionkey@0"
I20260808 23:26:53.700317 11957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance:
uuid: "1fe6b02094dd4d8aa91c77bf1a44299e"
format_stamp: "Formatted at 2026-08-08 23:26:53 on dist-test-slave-6bbx"
server_key: "ed9b3dac69d7cb4223ac9daefb8f5d02"
server_key_iv: "1ab2cb9392e0c5aade96c70524999abb"
server_key_version: "encryptionkey@0"
I20260808 23:26:53.705092 11957 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 23:26:53.708508 11983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:53.709731 11957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:26:53.709864 11957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "1fe6b02094dd4d8aa91c77bf1a44299e"
format_stamp: "Formatted at 2026-08-08 23:26:53 on dist-test-slave-6bbx"
server_key: "ed9b3dac69d7cb4223ac9daefb8f5d02"
server_key_iv: "1ab2cb9392e0c5aade96c70524999abb"
server_key_version: "encryptionkey@0"
I20260808 23:26:53.710017 11957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:26:53.746840 11957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:26:53.750862 11957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:26:53.751092 11957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:26:53.753605 11957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:26:53.753727 11957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:53.753839 11957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:26:53.753945 11957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 23:26:53 dist-test-slave-6bbx krb5kdc[11786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231613, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:26:53.775388 11998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:53.754194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:59933 (local address 127.9.44.126:44905)
0808 23:26:53.756560 (+ 2366us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:53.756567 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:53.756591 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:26:53.758331 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:53.758339 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:53.758432 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:53.758552 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:53.760635 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:53.761589 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:53.764044 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:53.764616 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:53.764729 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:53.769617 (+ 4888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:53.769623 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:53.769626 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:53.769659 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:53.772518 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:53.773193 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:53.773199 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:53.773202 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:53.773322 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:53.773740 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:53.773745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:53.773749 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:53.774016 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:53.774148 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:53.774803 (+ 655us) server_negotiation.cc:299] Negotiation successful
0808 23:26:53.775019 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2245,"thread_start_us":2142,"threads_started":1}
I20260808 23:26:53.776779 11999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:53.754849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:44905 (local address 127.9.44.65:59933)
0808 23:26:53.755819 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:53.755883 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:53.757984 (+ 2101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:53.758844 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:53.758853 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:53.759555 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:26:53.760439 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:53.760466 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:53.761747 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:53.761753 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:53.763899 (+ 2146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:53.763913 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:53.764889 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:53.764899 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:53.765137 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:53.765856 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:26:53.765893 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:26:53.769392 (+ 3499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:26:53.772648 (+ 3256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:53.772656 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:53.772674 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:53.773065 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:53.773434 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:53.773439 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:53.773443 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:53.773624 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:53.774145 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:26:53.774152 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:26:53.774458 (+ 306us) client_negotiation.cc:769] Sending connection context
0808 23:26:53.774686 (+ 228us) client_negotiation.cc:240] Negotiation successful
0808 23:26:53.775145 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":399,"threads_started":1}
I20260808 23:26:53.795243 11998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:53.782711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:43647 (local address 127.9.44.126:44905)
0808 23:26:53.782889 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:53.782896 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:53.783570 (+ 674us) server_negotiation.cc:407] Connection header received
0808 23:26:53.783715 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:53.783721 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:53.783799 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:53.783904 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:53.785119 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:53.786181 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:53.788056 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:53.788602 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:53.788711 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:53.790323 (+ 1612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:53.790328 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:53.790331 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:53.790367 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:53.792614 (+ 2247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:53.793409 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:53.793415 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:53.793419 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:53.793577 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:53.794034 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:53.794040 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:53.794044 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:53.794291 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:53.794411 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:53.794875 (+ 464us) server_negotiation.cc:299] Negotiation successful
0808 23:26:53.795029 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 23:26:53.795579 12097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:53.782693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:44905 (local address 127.9.44.65:43647)
0808 23:26:53.783469 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:53.783489 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:53.783591 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:53.783957 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:53.783965 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:53.784581 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:26:53.784965 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:53.784980 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:53.786328 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:53.786336 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:53.787913 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:53.787932 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:53.788788 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:53.788795 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:53.788947 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:53.789462 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:26:53.789477 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:26:53.790153 (+ 676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:26:53.792740 (+ 2587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:53.792745 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:53.792749 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:53.793289 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:53.793712 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:53.793717 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:53.793721 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:53.793906 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:53.794397 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:26:53.794402 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:26:53.794604 (+ 202us) client_negotiation.cc:769] Sending connection context
0808 23:26:53.794776 (+ 172us) client_negotiation.cc:240] Negotiation successful
0808 23:26:53.795118 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":490,"threads_started":1}
I20260808 23:26:53.804589 11957 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:39625
I20260808 23:26:53.804629 12155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:39625 every 8 connection(s)
I20260808 23:26:53.805692 11957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:26:53.807158 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 11957
I20260808 23:26:53.807363 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance
I20260808 23:26:53.807484 12157 heartbeater.cc:347] Connected to a master server at 127.9.44.126:44905
I20260808 23:26:53.807719 12157 heartbeater.cc:464] Registering TS with master...
I20260808 23:26:53.808075 9393 external_mini_cluster.cc:1428] Setting key c7b1178643fde1680986b784d1a57728
I20260808 23:26:53.808344 12157 heartbeater.cc:511] Master 127.9.44.126:44905 requested a full tablet report, sending...
I20260808 23:26:53.810400 11856 ts_manager.cc:194] Registered new tserver with Master: 1fe6b02094dd4d8aa91c77bf1a44299e (127.9.44.65:39625)
I20260808 23:26:53.811946 11856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.65@KRBTEST.COM'} at 127.9.44.65:59933
WARNING: no policy specified for kudu/127.9.44.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
I20260808 23:26:53.848174 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:0
--local_ip_for_outbound_sockets=127.9.44.66
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:44905
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:34267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:26:54.158583 12162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:26:54.158841 12162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:26:54.158980 12162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:26:54.166042 12162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:26:54.166158 12162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:26:54.166281 12162 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 23:26:54.166350 12162 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 23:26:54.166453 12162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:26:54.175592 12162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:34267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.44.126:44905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:26:54.176702 12162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:26:54.177922 12162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:26:54.189841 12172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:54.189941 12174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:54.189879 12176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:26:54.190834 12162 server_base.cc:1034] running on GCE node
I20260808 23:26:54.191437 12162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:26:54.192302 12162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:26:54.193501 12162 hybrid_clock.cc:648] HybridClock initialized: now 1786231614193441 us; error 51 us; skew 500 ppm
I20260808 23:26:54.193794 12162 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:26:54 dist-test-slave-6bbx krb5kdc[11786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231614, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:26:54.196676 12162 init.cc:377] Logged in from keytab as kudu/127.9.44.66@KRBTEST.COM (short username kudu)
I20260808 23:26:54.198174 12162 webserver.cc:460] Webserver started at http://127.9.44.66:34803/ using document root <none> and password file <none>
I20260808 23:26:54.198779 12162 fs_manager.cc:359] Metadata directory not provided
I20260808 23:26:54.198894 12162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:26:54.199174 12162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:26:54.201958 12162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-1/data/instance:
uuid: "e358b8bf6ce346458bbbeefd27838621"
format_stamp: "Formatted at 2026-08-08 23:26:54 on dist-test-slave-6bbx"
server_key: "9bddd310abb7d87f01d02bb4a42d00cc"
server_key_iv: "55c5ebd7f6220d9037c97aad150e8b5b"
server_key_version: "encryptionkey@0"
I20260808 23:26:54.202675 12162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance:
uuid: "e358b8bf6ce346458bbbeefd27838621"
format_stamp: "Formatted at 2026-08-08 23:26:54 on dist-test-slave-6bbx"
server_key: "9bddd310abb7d87f01d02bb4a42d00cc"
server_key_iv: "55c5ebd7f6220d9037c97aad150e8b5b"
server_key_version: "encryptionkey@0"
I20260808 23:26:54.207593 12162 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 23:26:54.210889 12186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:54.212128 12162 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:26:54.212276 12162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "e358b8bf6ce346458bbbeefd27838621"
format_stamp: "Formatted at 2026-08-08 23:26:54 on dist-test-slave-6bbx"
server_key: "9bddd310abb7d87f01d02bb4a42d00cc"
server_key_iv: "55c5ebd7f6220d9037c97aad150e8b5b"
server_key_version: "encryptionkey@0"
I20260808 23:26:54.212443 12162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:26:54.233170 12162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:26:54.236660 12162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:26:54.237028 12162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:26:54.239490 12162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:26:54.239594 12162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:54.239727 12162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:26:54.239806 12162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 23:26:54 dist-test-slave-6bbx krb5kdc[11786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231614, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:26:54.261492 11998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:54.240110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:35955 (local address 127.9.44.126:44905)
0808 23:26:54.240320 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:54.240326 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:54.241667 (+ 1341us) server_negotiation.cc:407] Connection header received
0808 23:26:54.244478 (+ 2811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:54.244484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:54.244598 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:54.244701 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:54.246682 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.247648 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.249940 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.250585 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.250725 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:54.255256 (+ 4531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:54.255264 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:54.255269 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:54.255304 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:54.258213 (+ 2909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:54.259142 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:54.259149 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:54.259152 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:54.259315 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:54.259829 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:54.259835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:54.259839 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:54.260123 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:54.260257 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:54.260921 (+ 664us) server_negotiation.cc:299] Negotiation successful
0808 23:26:54.261146 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260808 23:26:54.262398 12200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:54.240612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:44905 (local address 127.9.44.66:35955)
0808 23:26:54.241482 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:54.241544 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:54.244120 (+ 2576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:54.244961 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:54.244970 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:54.245647 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:26:54.246476 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:54.246506 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.247833 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.247842 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:54.249777 (+ 1935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:54.249798 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.250841 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.250849 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:54.251131 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:54.251791 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:26:54.251836 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:26:54.255045 (+ 3209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:26:54.258343 (+ 3298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:54.258355 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:54.258381 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:54.259013 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:54.259446 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:54.259460 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:54.259463 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:54.259711 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:54.260249 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:26:54.260256 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:26:54.260568 (+ 312us) client_negotiation.cc:769] Sending connection context
0808 23:26:54.260789 (+ 221us) client_negotiation.cc:240] Negotiation successful
0808 23:26:54.261246 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":345,"threads_started":1}
I20260808 23:26:54.280875 12162 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:41849
I20260808 23:26:54.280952 12361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:41849 every 8 connection(s)
I20260808 23:26:54.282414 12162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:26:54.282848 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 12162
I20260808 23:26:54.283015 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance
I20260808 23:26:54.283423 12316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:54.269124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:44905 (local address 127.9.44.66:45433)
0808 23:26:54.269733 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:54.269755 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:54.269898 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:54.270257 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:54.270266 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:54.270787 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:26:54.271054 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:54.271062 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.272458 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.272462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:54.273753 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:54.273770 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.274878 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.274887 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:54.275088 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:54.275757 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:26:54.275783 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:26:54.276892 (+ 1109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:26:54.280579 (+ 3687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:54.280584 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:54.280589 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:54.281194 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:54.281594 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:54.281597 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:54.281600 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:54.281717 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:54.282353 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:26:54.282356 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:26:54.282555 (+ 199us) client_negotiation.cc:769] Sending connection context
0808 23:26:54.282706 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 23:26:54.283012 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":342,"threads_started":1}
I20260808 23:26:54.283421 11998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:54.269064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:45433 (local address 127.9.44.126:44905)
0808 23:26:54.269250 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:54.269256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:54.269836 (+ 580us) server_negotiation.cc:407] Connection header received
0808 23:26:54.270023 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:54.270029 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:54.270107 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:54.270211 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:54.271186 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.272344 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.273931 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.274577 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.274713 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:54.277086 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:54.277093 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:54.277098 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:54.277135 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:54.280457 (+ 3322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:54.281317 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:54.281323 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:54.281326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:54.281489 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:54.281821 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:54.281827 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:54.281830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:54.282245 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:54.282398 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:54.282835 (+ 437us) server_negotiation.cc:299] Negotiation successful
0808 23:26:54.283086 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 23:26:54.283540 9393 external_mini_cluster.cc:1428] Setting key b1f7f93a819df2552bfa019e8e072ae6
I20260808 23:26:54.285929 12365 heartbeater.cc:347] Connected to a master server at 127.9.44.126:44905
I20260808 23:26:54.286231 12365 heartbeater.cc:464] Registering TS with master...
I20260808 23:26:54.287076 12365 heartbeater.cc:511] Master 127.9.44.126:44905 requested a full tablet report, sending...
I20260808 23:26:54.289042 11857 ts_manager.cc:194] Registered new tserver with Master: e358b8bf6ce346458bbbeefd27838621 (127.9.44.66:41849)
I20260808 23:26:54.289865 11857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.66@KRBTEST.COM'} at 127.9.44.66:35955
WARNING: no policy specified for kudu/127.9.44.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
I20260808 23:26:54.324682 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:0
--local_ip_for_outbound_sockets=127.9.44.67
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:44905
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:34267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:26:54.635160 12369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:26:54.635422 12369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:26:54.635569 12369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:26:54.642519 12369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:26:54.642640 12369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:26:54.642760 12369 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 23:26:54.642817 12369 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 23:26:54.642884 12369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:26:54.652208 12369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:34267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.44.126:44905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:26:54.653297 12369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:26:54.654512 12369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:26:54.666738 12375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:54.666690 12376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:26:54.666738 12378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:26:54.667622 12369 server_base.cc:1034] running on GCE node
I20260808 23:26:54.668252 12369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:26:54.669055 12369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:26:54.670261 12369 hybrid_clock.cc:648] HybridClock initialized: now 1786231614670212 us; error 36 us; skew 500 ppm
I20260808 23:26:54.670542 12369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:26:54 dist-test-slave-6bbx krb5kdc[11786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231614, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:26:54.673488 12369 init.cc:377] Logged in from keytab as kudu/127.9.44.67@KRBTEST.COM (short username kudu)
I20260808 23:26:54.674772 12369 webserver.cc:460] Webserver started at http://127.9.44.67:44693/ using document root <none> and password file <none>
I20260808 23:26:54.675377 12369 fs_manager.cc:359] Metadata directory not provided
I20260808 23:26:54.675489 12369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:26:54.675872 12369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:26:54.678553 12369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-2/data/instance:
uuid: "cbe141c1a3b7472bbdca35ea0f1c849b"
format_stamp: "Formatted at 2026-08-08 23:26:54 on dist-test-slave-6bbx"
server_key: "6f79069bd96bc688d4ca48dd56b64427"
server_key_iv: "630866b1c60a0df38aa2e84f604af57e"
server_key_version: "encryptionkey@0"
I20260808 23:26:54.679240 12369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance:
uuid: "cbe141c1a3b7472bbdca35ea0f1c849b"
format_stamp: "Formatted at 2026-08-08 23:26:54 on dist-test-slave-6bbx"
server_key: "6f79069bd96bc688d4ca48dd56b64427"
server_key_iv: "630866b1c60a0df38aa2e84f604af57e"
server_key_version: "encryptionkey@0"
I20260808 23:26:54.684028 12369 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.007s
I20260808 23:26:54.687141 12390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:26:54.688380 12369 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 23:26:54.688524 12369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "cbe141c1a3b7472bbdca35ea0f1c849b"
format_stamp: "Formatted at 2026-08-08 23:26:54 on dist-test-slave-6bbx"
server_key: "6f79069bd96bc688d4ca48dd56b64427"
server_key_iv: "630866b1c60a0df38aa2e84f604af57e"
server_key_version: "encryptionkey@0"
I20260808 23:26:54.688691 12369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:26:54.706061 12369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:26:54.709690 12369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:26:54.709933 12369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:26:54.712553 12369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:26:54.712715 12369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260808 23:26:54.712915 12369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:26:54.713014 12369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 23:26:54 dist-test-slave-6bbx krb5kdc[11786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231614, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:26:54.733240 11998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:54.712846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:34225 (local address 127.9.44.126:44905)
0808 23:26:54.713044 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:54.713050 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:54.714431 (+ 1381us) server_negotiation.cc:407] Connection header received
0808 23:26:54.716547 (+ 2116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:54.716553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:54.716629 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:54.716726 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:54.718699 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.719691 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.721998 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.722422 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.722534 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:54.727142 (+ 4608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:54.727147 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:54.727150 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:54.727182 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:54.729966 (+ 2784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:54.730918 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:54.730924 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:54.730927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:54.731057 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:54.731468 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:54.731474 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:54.731477 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:54.731833 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:54.732016 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:54.732652 (+ 636us) server_negotiation.cc:299] Negotiation successful
0808 23:26:54.732887 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 23:26:54.734107 12402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:54.713301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:44905 (local address 127.9.44.67:34225)
0808 23:26:54.714261 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:54.714312 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:54.716258 (+ 1946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:54.716964 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:54.716972 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:54.717639 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:26:54.718455 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:54.718485 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.719869 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.719875 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:54.721855 (+ 1980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:54.721868 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.722645 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.722654 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:54.722914 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:54.723613 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:26:54.723688 (+ 75us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:26:54.726944 (+ 3256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:26:54.730133 (+ 3189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:54.730142 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:54.730157 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:54.730747 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:54.731167 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:54.731172 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:54.731176 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:54.731357 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:54.731963 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:26:54.731970 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:26:54.732295 (+ 325us) client_negotiation.cc:769] Sending connection context
0808 23:26:54.732539 (+ 244us) client_negotiation.cc:240] Negotiation successful
0808 23:26:54.733009 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":354,"threads_started":1}
I20260808 23:26:54.752353 11998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:54.739543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:33513 (local address 127.9.44.126:44905)
0808 23:26:54.739747 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:54.739753 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:54.740203 (+ 450us) server_negotiation.cc:407] Connection header received
0808 23:26:54.740334 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:54.740340 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:54.740429 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:54.740546 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:54.741404 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.742521 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.743888 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.744386 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.744500 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:54.746665 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:54.746670 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:54.746674 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:54.746701 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:54.749670 (+ 2969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:54.750415 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:54.750422 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:54.750425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:54.750578 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:54.751062 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:54.751065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:54.751067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:54.751313 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:54.751420 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:54.751937 (+ 517us) server_negotiation.cc:299] Negotiation successful
0808 23:26:54.752125 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 23:26:54.752643 12486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:54.739489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:44905 (local address 127.9.44.67:33513)
0808 23:26:54.740120 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:54.740136 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:54.740243 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:54.740550 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:54.740553 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:54.741004 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:26:54.741282 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:54.741291 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.742648 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.742651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:54.743723 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:54.743743 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.744749 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.744757 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:54.745021 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:54.745631 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:26:54.745653 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:26:54.746482 (+ 829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:26:54.749811 (+ 3329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:54.749816 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:54.749820 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:54.750292 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:54.750732 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:54.750737 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:54.750741 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:54.750931 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:54.751419 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:26:54.751424 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:26:54.751631 (+ 207us) client_negotiation.cc:769] Sending connection context
0808 23:26:54.751841 (+ 210us) client_negotiation.cc:240] Negotiation successful
0808 23:26:54.752175 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":367,"threads_started":1}
I20260808 23:26:54.766407 12369 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:33875
I20260808 23:26:54.766445 12554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:33875 every 8 connection(s)
I20260808 23:26:54.767540 12369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:26:54.769460 12557 heartbeater.cc:347] Connected to a master server at 127.9.44.126:44905
I20260808 23:26:54.769665 12557 heartbeater.cc:464] Registering TS with master...
I20260808 23:26:54.770249 12557 heartbeater.cc:511] Master 127.9.44.126:44905 requested a full tablet report, sending...
I20260808 23:26:54.770337 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 12369
I20260808 23:26:54.770495 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance
I20260808 23:26:54.770917 9393 external_mini_cluster.cc:1428] Setting key 45532cb1f341eca2fee062f77c9c6e0d
I20260808 23:26:54.771562 11857 ts_manager.cc:194] Registered new tserver with Master: cbe141c1a3b7472bbdca35ea0f1c849b (127.9.44.67:33875)
I20260808 23:26:54.772650 11857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.67@KRBTEST.COM'} at 127.9.44.67:34225
I20260808 23:26:54.773834 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 23:26:54 dist-test-slave-6bbx krb5kdc[11786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
I20260808 23:26:54.814497 12157 heartbeater.cc:503] Master 127.9.44.126:44905 was elected leader, sending a full tablet report...
Aug 08 23:26:54 dist-test-slave-6bbx krb5kdc[11786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:26:54.821712 11998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:54.808382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49584 (local address 127.9.44.126:44905)
0808 23:26:54.808569 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:54.808575 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:54.808812 (+ 237us) server_negotiation.cc:407] Connection header received
0808 23:26:54.808983 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:54.808987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:54.809048 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:54.809128 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:54.810911 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.811633 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.812766 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.813218 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.813304 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:54.817297 (+ 3993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:54.817302 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:54.817305 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:54.817336 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:54.819763 (+ 2427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:54.820260 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:54.820263 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:54.820265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:54.820378 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:54.820644 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:54.820647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:54.820649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:54.820882 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:54.821068 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:54.821246 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 23:26:54.821399 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 23:26:54.825377 11856 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49584:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:26:54.832474 11856 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:26:54.878762 12579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:54.845002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32848 (local address 127.9.44.66:41849)
0808 23:26:54.868175 (+ 23173us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:54.868185 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:54.871098 (+ 2913us) server_negotiation.cc:407] Connection header received
0808 23:26:54.871279 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:54.871287 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:54.871573 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:54.871797 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:26:54.872523 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.874043 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.876401 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.877689 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.877805 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:54.878009 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:54.878279 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 23:26:54.878446 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22972,"thread_start_us":23183,"threads_started":1}
I20260808 23:26:54.878794 12573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:54.844502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:41849 (local address 127.0.0.1:32848)
0808 23:26:54.870976 (+ 26474us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:54.870989 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:54.871115 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:54.871802 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:54.871809 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:54.871837 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:26:54.872276 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:54.872293 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.874219 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.874224 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:54.876252 (+ 2028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:54.876264 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.877874 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.877880 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:54.878068 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:54.878105 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 23:26:54.878210 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 23:26:54.878376 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26383,"mutex_wait_us":83,"thread_start_us":24921,"threads_started":1}
I20260808 23:26:54.879911 12576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:54.844264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:33875 (local address 127.0.0.1:34976)
0808 23:26:54.870437 (+ 26173us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:54.870454 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:54.870560 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:54.872272 (+ 1712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:54.872277 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:54.872307 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:26:54.872863 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:54.872877 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.874642 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.874647 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:54.877224 (+ 2577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:54.877241 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.879092 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.879099 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:54.879294 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:54.879394 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 23:26:54.879476 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 23:26:54.879592 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26064,"mutex_wait_us":131,"thread_start_us":25919,"threads_started":1}
I20260808 23:26:54.880468 12575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:54.844059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48624 (local address 127.9.44.65:39625)
0808 23:26:54.871583 (+ 27524us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:54.871591 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:54.871619 (+ 28us) server_negotiation.cc:407] Connection header received
0808 23:26:54.871749 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:54.871758 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:54.871988 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:54.872210 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:26:54.873029 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.874508 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.877410 (+ 2902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.878826 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.879390 (+ 564us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:54.879602 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:54.880023 (+ 421us) server_negotiation.cc:299] Negotiation successful
0808 23:26:54.880183 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27368,"thread_start_us":28314,"threads_started":1}
I20260808 23:26:54.881155 12578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:54.844676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34976 (local address 127.9.44.67:33875)
0808 23:26:54.871501 (+ 26825us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:54.871507 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:54.871532 (+ 25us) server_negotiation.cc:407] Connection header received
0808 23:26:54.871603 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:54.871607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:54.871838 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:54.872623 (+ 785us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:26:54.873096 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.874508 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.877421 (+ 2913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.878826 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.880430 (+ 1604us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:54.880667 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:54.880735 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 23:26:54.880882 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26689,"thread_start_us":27308,"threads_started":1}
I20260808 23:26:54.882712 12256 tablet_service.cc:1467] Processing CreateTablet for tablet 8c09352fde7a468ca5ff13f695f7df42 (DEFAULT_TABLE table=test-table [id=ffedd157234a4d0886a7e0eda56392a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:26:54.884081 12577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:54.843874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:39625 (local address 127.0.0.1:48624)
0808 23:26:54.868998 (+ 25124us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:54.869043 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:54.869236 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:54.872203 (+ 2967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:54.872207 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:54.872239 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:26:54.872826 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:54.872845 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.874666 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.874671 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:54.877235 (+ 2564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:54.877245 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:54.879107 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:54.879113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:54.879298 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:54.879471 (+ 173us) client_negotiation.cc:769] Sending connection context
0808 23:26:54.879952 (+ 481us) client_negotiation.cc:240] Negotiation successful
0808 23:26:54.880070 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24831,"mutex_wait_us":176,"spinlock_wait_cycles":896,"thread_start_us":26887,"threads_started":1}
I20260808 23:26:54.884111 12256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c09352fde7a468ca5ff13f695f7df42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:54.887502 12057 tablet_service.cc:1467] Processing CreateTablet for tablet 8c09352fde7a468ca5ff13f695f7df42 (DEFAULT_TABLE table=test-table [id=ffedd157234a4d0886a7e0eda56392a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:26:54.887502 12461 tablet_service.cc:1467] Processing CreateTablet for tablet 8c09352fde7a468ca5ff13f695f7df42 (DEFAULT_TABLE table=test-table [id=ffedd157234a4d0886a7e0eda56392a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:26:54.888885 12461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c09352fde7a468ca5ff13f695f7df42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:54.888885 12057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c09352fde7a468ca5ff13f695f7df42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:54.894850 12580 tablet_bootstrap.cc:492] T 8c09352fde7a468ca5ff13f695f7df42 P e358b8bf6ce346458bbbeefd27838621: Bootstrap starting.
I20260808 23:26:54.897886 12580 tablet_bootstrap.cc:654] T 8c09352fde7a468ca5ff13f695f7df42 P e358b8bf6ce346458bbbeefd27838621: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:54.899084 12580 log.cc:824] T 8c09352fde7a468ca5ff13f695f7df42 P e358b8bf6ce346458bbbeefd27838621: Log is configured to *not* fsync() on all Append() calls
I20260808 23:26:54.901656 12580 tablet_bootstrap.cc:492] T 8c09352fde7a468ca5ff13f695f7df42 P e358b8bf6ce346458bbbeefd27838621: No bootstrap required, opened a new log
I20260808 23:26:54.901897 12580 ts_tablet_manager.cc:1397] T 8c09352fde7a468ca5ff13f695f7df42 P e358b8bf6ce346458bbbeefd27838621: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 23:26:54.902984 12582 tablet_bootstrap.cc:492] T 8c09352fde7a468ca5ff13f695f7df42 P cbe141c1a3b7472bbdca35ea0f1c849b: Bootstrap starting.
I20260808 23:26:54.906760 12582 tablet_bootstrap.cc:654] T 8c09352fde7a468ca5ff13f695f7df42 P cbe141c1a3b7472bbdca35ea0f1c849b: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:54.907972 12582 log.cc:824] T 8c09352fde7a468ca5ff13f695f7df42 P cbe141c1a3b7472bbdca35ea0f1c849b: Log is configured to *not* fsync() on all Append() calls
I20260808 23:26:54.910859 12582 tablet_bootstrap.cc:492] T 8c09352fde7a468ca5ff13f695f7df42 P cbe141c1a3b7472bbdca35ea0f1c849b: No bootstrap required, opened a new log
I20260808 23:26:54.911149 12582 ts_tablet_manager.cc:1397] T 8c09352fde7a468ca5ff13f695f7df42 P cbe141c1a3b7472bbdca35ea0f1c849b: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 23:26:54.911783 12584 tablet_bootstrap.cc:492] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e: Bootstrap starting.
I20260808 23:26:54.911685 12580 raft_consensus.cc:348] T 8c09352fde7a468ca5ff13f695f7df42 P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } }
I20260808 23:26:54.912110 12580 raft_consensus.cc:374] T 8c09352fde7a468ca5ff13f695f7df42 P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:54.912228 12580 raft_consensus.cc:729] T 8c09352fde7a468ca5ff13f695f7df42 P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e358b8bf6ce346458bbbeefd27838621, State: Initialized, Role: FOLLOWER
I20260808 23:26:54.912833 12580 consensus_queue.cc:260] T 8c09352fde7a468ca5ff13f695f7df42 P e358b8bf6ce346458bbbeefd27838621 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } }
I20260808 23:26:54.915081 12584 tablet_bootstrap.cc:654] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:54.916131 12584 log.cc:824] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e: Log is configured to *not* fsync() on all Append() calls
I20260808 23:26:54.920398 12365 heartbeater.cc:503] Master 127.9.44.126:44905 was elected leader, sending a full tablet report...
I20260808 23:26:54.920974 12580 ts_tablet_manager.cc:1428] T 8c09352fde7a468ca5ff13f695f7df42 P e358b8bf6ce346458bbbeefd27838621: Time spent starting tablet: real 0.019s user 0.012s sys 0.007s
I20260808 23:26:54.921644 12582 raft_consensus.cc:348] T 8c09352fde7a468ca5ff13f695f7df42 P cbe141c1a3b7472bbdca35ea0f1c849b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } }
I20260808 23:26:54.922055 12582 raft_consensus.cc:374] T 8c09352fde7a468ca5ff13f695f7df42 P cbe141c1a3b7472bbdca35ea0f1c849b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:54.922171 12582 raft_consensus.cc:729] T 8c09352fde7a468ca5ff13f695f7df42 P cbe141c1a3b7472bbdca35ea0f1c849b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbe141c1a3b7472bbdca35ea0f1c849b, State: Initialized, Role: FOLLOWER
I20260808 23:26:54.922709 12582 consensus_queue.cc:260] T 8c09352fde7a468ca5ff13f695f7df42 P cbe141c1a3b7472bbdca35ea0f1c849b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } }
I20260808 23:26:54.923633 12557 heartbeater.cc:503] Master 127.9.44.126:44905 was elected leader, sending a full tablet report...
I20260808 23:26:54.924278 12582 ts_tablet_manager.cc:1428] T 8c09352fde7a468ca5ff13f695f7df42 P cbe141c1a3b7472bbdca35ea0f1c849b: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260808 23:26:54.924518 12584 tablet_bootstrap.cc:492] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e: No bootstrap required, opened a new log
I20260808 23:26:54.924793 12584 ts_tablet_manager.cc:1397] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e: Time spent bootstrapping tablet: real 0.016s user 0.011s sys 0.000s
I20260808 23:26:54.932132 12584 raft_consensus.cc:348] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } }
I20260808 23:26:54.932435 12584 raft_consensus.cc:374] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:54.932507 12584 raft_consensus.cc:729] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fe6b02094dd4d8aa91c77bf1a44299e, State: Initialized, Role: FOLLOWER
I20260808 23:26:54.932976 12584 consensus_queue.cc:260] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } }
I20260808 23:26:54.934269 12584 ts_tablet_manager.cc:1428] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
W20260808 23:26:55.019050 12558 tablet.cc:2369] T 8c09352fde7a468ca5ff13f695f7df42 P cbe141c1a3b7472bbdca35ea0f1c849b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:26:55.039433 12366 tablet.cc:2369] T 8c09352fde7a468ca5ff13f695f7df42 P e358b8bf6ce346458bbbeefd27838621: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:26:55.057854 12159 tablet.cc:2369] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:26:55.173722 12590 raft_consensus.cc:482] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:26:55.174034 12590 raft_consensus.cc:504] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } }
I20260808 23:26:55.175740 12590 leader_election.cc:288] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cbe141c1a3b7472bbdca35ea0f1c849b (127.9.44.67:33875), e358b8bf6ce346458bbbeefd27838621 (127.9.44.66:41849)
I20260808 23:26:55.182013 12579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:55.176174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:38141 (local address 127.9.44.66:41849)
0808 23:26:55.176411 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:55.176417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:55.176843 (+ 426us) server_negotiation.cc:407] Connection header received
0808 23:26:55.177051 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:55.177057 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:55.177120 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:55.177194 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:26:55.177755 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:55.178454 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:55.180405 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:55.181151 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:55.181270 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:55.181371 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:55.181628 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 23:26:55.181759 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:26:55.182112 12591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:55.176064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:41849 (local address 127.9.44.65:38141)
0808 23:26:55.176735 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:55.176751 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:55.176907 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:55.177240 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:55.177244 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:55.177337 (+ 93us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:26:55.177613 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:55.177622 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:55.178580 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:55.178584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:55.180256 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:55.180272 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:55.181358 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:55.181364 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:55.181484 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:55.181507 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 23:26:55.181598 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 23:26:55.181722 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"mutex_wait_us":60,"spinlock_wait_cycles":23808,"thread_start_us":616,"threads_started":1}
I20260808 23:26:55.182309 12578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:55.176224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:37033 (local address 127.9.44.67:33875)
0808 23:26:55.176406 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:55.176412 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:55.177246 (+ 834us) server_negotiation.cc:407] Connection header received
0808 23:26:55.177428 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:55.177433 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:55.177501 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:55.177576 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:26:55.178102 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:55.178822 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:55.180660 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:55.181416 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:55.181527 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:55.181630 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:55.181933 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 23:26:55.182058 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 23:26:55.182705 12592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:55.176450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:33875 (local address 127.9.44.65:37033)
0808 23:26:55.177141 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:55.177155 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:55.177285 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:55.177623 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:55.177627 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:55.177651 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:26:55.177957 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:55.177966 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:55.178978 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:55.178983 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:55.180502 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:55.180516 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:55.181630 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:55.181637 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:55.181772 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:55.181795 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 23:26:55.181884 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 23:26:55.182009 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"mutex_wait_us":2,"thread_start_us":458,"threads_started":1}
I20260808 23:26:55.183406 12289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c09352fde7a468ca5ff13f695f7df42" candidate_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e358b8bf6ce346458bbbeefd27838621" is_pre_election: true
I20260808 23:26:55.183593 12492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c09352fde7a468ca5ff13f695f7df42" candidate_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" is_pre_election: true
I20260808 23:26:55.184036 12289 raft_consensus.cc:2454] T 8c09352fde7a468ca5ff13f695f7df42 P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fe6b02094dd4d8aa91c77bf1a44299e in term 0.
I20260808 23:26:55.184202 12492 raft_consensus.cc:2454] T 8c09352fde7a468ca5ff13f695f7df42 P cbe141c1a3b7472bbdca35ea0f1c849b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fe6b02094dd4d8aa91c77bf1a44299e in term 0.
I20260808 23:26:55.184788 11984 leader_election.cc:302] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1fe6b02094dd4d8aa91c77bf1a44299e, cbe141c1a3b7472bbdca35ea0f1c849b, e358b8bf6ce346458bbbeefd27838621; no voters:
I20260808 23:26:55.185171 12590 raft_consensus.cc:2781] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:26:55.185319 12590 raft_consensus.cc:482] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:26:55.185423 12590 raft_consensus.cc:3037] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:55.187641 12590 raft_consensus.cc:504] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } }
I20260808 23:26:55.188297 12590 leader_election.cc:288] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e [CANDIDATE]: Term 1 election: Requested vote from peers cbe141c1a3b7472bbdca35ea0f1c849b (127.9.44.67:33875), e358b8bf6ce346458bbbeefd27838621 (127.9.44.66:41849)
I20260808 23:26:55.188628 12492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c09352fde7a468ca5ff13f695f7df42" candidate_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b"
I20260808 23:26:55.188795 12492 raft_consensus.cc:3037] T 8c09352fde7a468ca5ff13f695f7df42 P cbe141c1a3b7472bbdca35ea0f1c849b [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:55.188750 12289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c09352fde7a468ca5ff13f695f7df42" candidate_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e358b8bf6ce346458bbbeefd27838621"
I20260808 23:26:55.188939 12289 raft_consensus.cc:3037] T 8c09352fde7a468ca5ff13f695f7df42 P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:55.190850 12492 raft_consensus.cc:2454] T 8c09352fde7a468ca5ff13f695f7df42 P cbe141c1a3b7472bbdca35ea0f1c849b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fe6b02094dd4d8aa91c77bf1a44299e in term 1.
I20260808 23:26:55.191164 12289 raft_consensus.cc:2454] T 8c09352fde7a468ca5ff13f695f7df42 P e358b8bf6ce346458bbbeefd27838621 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fe6b02094dd4d8aa91c77bf1a44299e in term 1.
I20260808 23:26:55.191191 11991 leader_election.cc:302] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fe6b02094dd4d8aa91c77bf1a44299e, cbe141c1a3b7472bbdca35ea0f1c849b; no voters:
I20260808 23:26:55.191690 12590 raft_consensus.cc:2781] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:26:55.192070 12590 raft_consensus.cc:686] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 1 LEADER]: Becoming Leader. State: Replica: 1fe6b02094dd4d8aa91c77bf1a44299e, State: Running, Role: LEADER
I20260808 23:26:55.192555 12590 consensus_queue.cc:237] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } }
I20260808 23:26:55.196832 11857 catalog_manager.cc:5495] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fe6b02094dd4d8aa91c77bf1a44299e (127.9.44.65). New cstate: current_term: 1 leader_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } health_report { overall_health: UNKNOWN } } }
Aug 08 23:26:55 dist-test-slave-6bbx krb5kdc[11786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231613, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:26:55.269801 12601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:55.257798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:44905 (local address 127.0.0.1:49592)
0808 23:26:55.258312 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:55.258327 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:55.258438 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:55.258695 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:55.258698 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:55.259343 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:26:55.259603 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:55.259613 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:55.260574 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:55.260577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:55.261524 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:55.261535 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:55.262211 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:55.262215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:55.262335 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:55.262895 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:26:55.262917 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:26:55.265048 (+ 2131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:26:55.267714 (+ 2666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:55.267725 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:55.267730 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:55.268103 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:55.268427 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:26:55.268430 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:26:55.268433 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:26:55.268567 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:26:55.269018 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:26:55.269024 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:26:55.269167 (+ 143us) client_negotiation.cc:769] Sending connection context
0808 23:26:55.269291 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 23:26:55.269481 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":281,"threads_started":1}
I20260808 23:26:55.269902 11998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:55.257765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49592 (local address 127.9.44.126:44905)
0808 23:26:55.257925 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:55.257932 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:55.258400 (+ 468us) server_negotiation.cc:407] Connection header received
0808 23:26:55.258522 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:55.258526 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:55.258583 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:55.258657 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:55.259757 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:55.260471 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:55.261638 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:55.262051 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:55.262152 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:55.265193 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:55.265198 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:55.265201 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:55.265232 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:55.267573 (+ 2341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:55.268208 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:55.268214 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:55.268216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:55.268336 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:55.268661 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:55.268664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:55.268666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:55.268914 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:55.269038 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:55.269393 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 23:26:55.269592 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":46}
I20260808 23:26:55.274343 11857 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:49592:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 23:26:55.275363 11857 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:26:55.281430 12057 tablet_service.cc:1467] Processing CreateTablet for tablet dfcd92eed5924938a1e74638e2242614 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=62a5c6f1a7134161b48523328087d7e9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 23:26:55.281589 12256 tablet_service.cc:1467] Processing CreateTablet for tablet dfcd92eed5924938a1e74638e2242614 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=62a5c6f1a7134161b48523328087d7e9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 23:26:55.281874 12461 tablet_service.cc:1467] Processing CreateTablet for tablet dfcd92eed5924938a1e74638e2242614 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=62a5c6f1a7134161b48523328087d7e9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 23:26:55.282415 12057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfcd92eed5924938a1e74638e2242614. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:55.282527 12256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfcd92eed5924938a1e74638e2242614. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:55.282636 12461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfcd92eed5924938a1e74638e2242614. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:55.289306 12582 tablet_bootstrap.cc:492] T dfcd92eed5924938a1e74638e2242614 P cbe141c1a3b7472bbdca35ea0f1c849b: Bootstrap starting.
I20260808 23:26:55.289309 12584 tablet_bootstrap.cc:492] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e: Bootstrap starting.
I20260808 23:26:55.289584 12580 tablet_bootstrap.cc:492] T dfcd92eed5924938a1e74638e2242614 P e358b8bf6ce346458bbbeefd27838621: Bootstrap starting.
I20260808 23:26:55.290869 12582 tablet_bootstrap.cc:654] T dfcd92eed5924938a1e74638e2242614 P cbe141c1a3b7472bbdca35ea0f1c849b: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:55.290869 12584 tablet_bootstrap.cc:654] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:55.291154 12580 tablet_bootstrap.cc:654] T dfcd92eed5924938a1e74638e2242614 P e358b8bf6ce346458bbbeefd27838621: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:55.299010 12580 tablet_bootstrap.cc:492] T dfcd92eed5924938a1e74638e2242614 P e358b8bf6ce346458bbbeefd27838621: No bootstrap required, opened a new log
I20260808 23:26:55.299118 12584 tablet_bootstrap.cc:492] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e: No bootstrap required, opened a new log
I20260808 23:26:55.299155 12580 ts_tablet_manager.cc:1397] T dfcd92eed5924938a1e74638e2242614 P e358b8bf6ce346458bbbeefd27838621: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260808 23:26:55.299271 12584 ts_tablet_manager.cc:1397] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20260808 23:26:55.300062 12582 tablet_bootstrap.cc:492] T dfcd92eed5924938a1e74638e2242614 P cbe141c1a3b7472bbdca35ea0f1c849b: No bootstrap required, opened a new log
I20260808 23:26:55.299976 12584 raft_consensus.cc:348] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:55.299942 12580 raft_consensus.cc:348] T dfcd92eed5924938a1e74638e2242614 P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:55.300164 12584 raft_consensus.cc:374] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:55.300168 12580 raft_consensus.cc:374] T dfcd92eed5924938a1e74638e2242614 P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:55.300212 12582 ts_tablet_manager.cc:1397] T dfcd92eed5924938a1e74638e2242614 P cbe141c1a3b7472bbdca35ea0f1c849b: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260808 23:26:55.300251 12584 raft_consensus.cc:729] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fe6b02094dd4d8aa91c77bf1a44299e, State: Initialized, Role: FOLLOWER
I20260808 23:26:55.300254 12580 raft_consensus.cc:729] T dfcd92eed5924938a1e74638e2242614 P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e358b8bf6ce346458bbbeefd27838621, State: Initialized, Role: FOLLOWER
I20260808 23:26:55.300503 12584 consensus_queue.cc:260] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:55.300480 12580 consensus_queue.cc:260] T dfcd92eed5924938a1e74638e2242614 P e358b8bf6ce346458bbbeefd27838621 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:55.300899 12582 raft_consensus.cc:348] T dfcd92eed5924938a1e74638e2242614 P cbe141c1a3b7472bbdca35ea0f1c849b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:55.300905 12590 tablet_replica.cc:439] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } } }
I20260808 23:26:55.301079 12584 ts_tablet_manager.cc:1428] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 23:26:55.301131 12582 raft_consensus.cc:374] T dfcd92eed5924938a1e74638e2242614 P cbe141c1a3b7472bbdca35ea0f1c849b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:55.300925 12587 tablet_replica.cc:439] T dfcd92eed5924938a1e74638e2242614 P e358b8bf6ce346458bbbeefd27838621: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } } }
I20260808 23:26:55.301187 12590 tablet_replica.cc:442] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:26:55.301225 12582 raft_consensus.cc:729] T dfcd92eed5924938a1e74638e2242614 P cbe141c1a3b7472bbdca35ea0f1c849b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbe141c1a3b7472bbdca35ea0f1c849b, State: Initialized, Role: FOLLOWER
I20260808 23:26:55.301244 12587 tablet_replica.cc:442] T dfcd92eed5924938a1e74638e2242614 P e358b8bf6ce346458bbbeefd27838621: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:26:55.301443 12582 consensus_queue.cc:260] T dfcd92eed5924938a1e74638e2242614 P cbe141c1a3b7472bbdca35ea0f1c849b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:55.302038 12582 ts_tablet_manager.cc:1428] T dfcd92eed5924938a1e74638e2242614 P cbe141c1a3b7472bbdca35ea0f1c849b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260808 23:26:55.301903 12589 tablet_replica.cc:439] T dfcd92eed5924938a1e74638e2242614 P cbe141c1a3b7472bbdca35ea0f1c849b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } } }
I20260808 23:26:55.302242 12589 tablet_replica.cc:442] T dfcd92eed5924938a1e74638e2242614 P cbe141c1a3b7472bbdca35ea0f1c849b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:26:55.302752 12580 ts_tablet_manager.cc:1428] T dfcd92eed5924938a1e74638e2242614 P e358b8bf6ce346458bbbeefd27838621: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 23:26:55.459156 12590 raft_consensus.cc:482] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:26:55.459375 12590 raft_consensus.cc:504] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:55.459998 12590 leader_election.cc:288] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e358b8bf6ce346458bbbeefd27838621 (127.9.44.66:41849), cbe141c1a3b7472bbdca35ea0f1c849b (127.9.44.67:33875)
I20260808 23:26:55.460635 12492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfcd92eed5924938a1e74638e2242614" candidate_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" is_pre_election: true
I20260808 23:26:55.460635 12289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfcd92eed5924938a1e74638e2242614" candidate_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e358b8bf6ce346458bbbeefd27838621" is_pre_election: true
I20260808 23:26:55.460916 12492 raft_consensus.cc:2454] T dfcd92eed5924938a1e74638e2242614 P cbe141c1a3b7472bbdca35ea0f1c849b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fe6b02094dd4d8aa91c77bf1a44299e in term 0.
I20260808 23:26:55.460922 12289 raft_consensus.cc:2454] T dfcd92eed5924938a1e74638e2242614 P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fe6b02094dd4d8aa91c77bf1a44299e in term 0.
I20260808 23:26:55.461292 11991 leader_election.cc:302] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fe6b02094dd4d8aa91c77bf1a44299e, cbe141c1a3b7472bbdca35ea0f1c849b; no voters:
I20260808 23:26:55.461570 12590 raft_consensus.cc:2781] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:26:55.461661 12590 raft_consensus.cc:482] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:26:55.461740 12590 raft_consensus.cc:3037] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:55.463644 12590 raft_consensus.cc:504] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:55.464191 12590 leader_election.cc:288] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e [CANDIDATE]: Term 1 election: Requested vote from peers e358b8bf6ce346458bbbeefd27838621 (127.9.44.66:41849), cbe141c1a3b7472bbdca35ea0f1c849b (127.9.44.67:33875)
I20260808 23:26:55.464556 12289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfcd92eed5924938a1e74638e2242614" candidate_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e358b8bf6ce346458bbbeefd27838621"
I20260808 23:26:55.464586 12492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfcd92eed5924938a1e74638e2242614" candidate_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b"
I20260808 23:26:55.464736 12289 raft_consensus.cc:3037] T dfcd92eed5924938a1e74638e2242614 P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:55.464776 12492 raft_consensus.cc:3037] T dfcd92eed5924938a1e74638e2242614 P cbe141c1a3b7472bbdca35ea0f1c849b [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:55.466634 12492 raft_consensus.cc:2454] T dfcd92eed5924938a1e74638e2242614 P cbe141c1a3b7472bbdca35ea0f1c849b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fe6b02094dd4d8aa91c77bf1a44299e in term 1.
I20260808 23:26:55.466713 12289 raft_consensus.cc:2454] T dfcd92eed5924938a1e74638e2242614 P e358b8bf6ce346458bbbeefd27838621 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fe6b02094dd4d8aa91c77bf1a44299e in term 1.
I20260808 23:26:55.467020 11991 leader_election.cc:302] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fe6b02094dd4d8aa91c77bf1a44299e, cbe141c1a3b7472bbdca35ea0f1c849b; no voters:
I20260808 23:26:55.467274 12590 raft_consensus.cc:2781] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:26:55.467427 12590 raft_consensus.cc:686] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 1 LEADER]: Becoming Leader. State: Replica: 1fe6b02094dd4d8aa91c77bf1a44299e, State: Running, Role: LEADER
I20260808 23:26:55.467720 12590 consensus_queue.cc:237] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:55.469273 12593 tablet_replica.cc:439] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e: TxnStatusTablet state changed. Reason: New leader 1fe6b02094dd4d8aa91c77bf1a44299e. Latest consensus state: current_term: 1 leader_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } } }
I20260808 23:26:55.469537 12593 tablet_replica.cc:442] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e: This TxnStatusTablet replica's current role is: LEADER
I20260808 23:26:55.470134 12607 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 23:26:55.470404 12607 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 23:26:55.470561 11853 catalog_manager.cc:5495] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fe6b02094dd4d8aa91c77bf1a44299e (127.9.44.65). New cstate: current_term: 1 leader_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } health_report { overall_health: UNKNOWN } } }
I20260808 23:26:55.498337 12614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:55.491643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48632 (local address 127.9.44.65:39625)
0808 23:26:55.492542 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:55.492546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:55.492565 (+ 19us) server_negotiation.cc:407] Connection header received
0808 23:26:55.492637 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:55.492640 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:55.492720 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:55.492836 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:26:55.493293 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:55.494152 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:55.496057 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:55.496491 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:55.496584 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:55.497145 (+ 561us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:26:55.497717 (+ 572us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:26:55.497817 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:55.497980 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 23:26:55.498043 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":791,"thread_start_us":651,"threads_started":1}
I20260808 23:26:55.498409 12601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:55.491569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:39625 (local address 127.0.0.1:48632)
0808 23:26:55.491837 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:55.491860 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:55.492003 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:55.492898 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:55.492903 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:55.492917 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 23:26:55.493113 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:55.493125 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:55.494290 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:55.494296 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:55.495920 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:55.495937 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:55.496643 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:55.496650 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:55.496840 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:55.496931 (+ 91us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 23:26:55.497828 (+ 897us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 23:26:55.497852 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 23:26:55.497946 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 23:26:55.498103 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 23:26:55.505848 12492 raft_consensus.cc:1261] T dfcd92eed5924938a1e74638e2242614 P cbe141c1a3b7472bbdca35ea0f1c849b [term 1 FOLLOWER]: Refusing update from remote peer 1fe6b02094dd4d8aa91c77bf1a44299e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:26:55.505900 12289 raft_consensus.cc:1261] T dfcd92eed5924938a1e74638e2242614 P e358b8bf6ce346458bbbeefd27838621 [term 1 FOLLOWER]: Refusing update from remote peer 1fe6b02094dd4d8aa91c77bf1a44299e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:26:55.506832 12593 consensus_queue.cc:1035] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:26:55.507141 12590 consensus_queue.cc:1035] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:26:55.522966 12587 tablet_replica.cc:439] T dfcd92eed5924938a1e74638e2242614 P e358b8bf6ce346458bbbeefd27838621: TxnStatusTablet state changed. Reason: New leader 1fe6b02094dd4d8aa91c77bf1a44299e. Latest consensus state: current_term: 1 leader_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } } }
I20260808 23:26:55.523258 12587 tablet_replica.cc:442] T dfcd92eed5924938a1e74638e2242614 P e358b8bf6ce346458bbbeefd27838621: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:26:55.525125 12589 tablet_replica.cc:439] T dfcd92eed5924938a1e74638e2242614 P cbe141c1a3b7472bbdca35ea0f1c849b: TxnStatusTablet state changed. Reason: New leader 1fe6b02094dd4d8aa91c77bf1a44299e. Latest consensus state: current_term: 1 leader_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } } }
I20260808 23:26:55.525389 12589 tablet_replica.cc:442] T dfcd92eed5924938a1e74638e2242614 P cbe141c1a3b7472bbdca35ea0f1c849b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:26:55.526628 12593 tablet_replica.cc:439] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } } }
I20260808 23:26:55.526880 12593 tablet_replica.cc:442] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e: This TxnStatusTablet replica's current role is: LEADER
I20260808 23:26:55.527642 12593 tablet_replica.cc:439] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } } }
I20260808 23:26:55.527930 12593 tablet_replica.cc:442] T dfcd92eed5924938a1e74638e2242614 P 1fe6b02094dd4d8aa91c77bf1a44299e: This TxnStatusTablet replica's current role is: LEADER
I20260808 23:26:55.530493 12587 tablet_replica.cc:439] T dfcd92eed5924938a1e74638e2242614 P e358b8bf6ce346458bbbeefd27838621: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } } }
I20260808 23:26:55.530723 12587 tablet_replica.cc:442] T dfcd92eed5924938a1e74638e2242614 P e358b8bf6ce346458bbbeefd27838621: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:26:55.534055 12589 tablet_replica.cc:439] T dfcd92eed5924938a1e74638e2242614 P cbe141c1a3b7472bbdca35ea0f1c849b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } } }
I20260808 23:26:55.534272 12589 tablet_replica.cc:442] T dfcd92eed5924938a1e74638e2242614 P cbe141c1a3b7472bbdca35ea0f1c849b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:26:55.546494 12614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:55.540570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48636 (local address 127.9.44.65:39625)
0808 23:26:55.540749 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:55.540756 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:55.541276 (+ 520us) server_negotiation.cc:407] Connection header received
0808 23:26:55.541433 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:55.541439 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:55.541529 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:55.541626 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:26:55.542001 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:55.543246 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:55.544692 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:55.545369 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:55.545492 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:55.545811 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:26:55.545938 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:26:55.546033 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:55.546109 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 23:26:55.546185 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 23:26:55.563611 12614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:55.557445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:36113 (local address 127.9.44.65:39625)
0808 23:26:55.557599 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:55.557602 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:55.558161 (+ 559us) server_negotiation.cc:407] Connection header received
0808 23:26:55.558317 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:55.558326 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:55.558402 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:55.558469 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:26:55.559031 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:55.560206 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:55.561594 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:55.562267 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:55.562402 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:55.562844 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:26:55.562962 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:26:55.563046 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:55.563167 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 23:26:55.563242 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 23:26:55.563612 12632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:55.557422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:39625 (local address 127.9.44.65:36113)
0808 23:26:55.558068 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:55.558086 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:55.558211 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:55.558512 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:55.558516 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:55.558527 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 23:26:55.558851 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:55.558863 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:55.560331 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:55.560335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:55.561458 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:55.561470 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:55.562441 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:55.562445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:55.562590 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:55.562644 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 23:26:55.563066 (+ 422us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 23:26:55.563074 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 23:26:55.563129 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 23:26:55.563222 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":349,"threads_started":1}
I20260808 23:26:55.572508 12492 raft_consensus.cc:1261] T 8c09352fde7a468ca5ff13f695f7df42 P cbe141c1a3b7472bbdca35ea0f1c849b [term 1 FOLLOWER]: Refusing update from remote peer 1fe6b02094dd4d8aa91c77bf1a44299e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:26:55.572682 12288 raft_consensus.cc:1261] T 8c09352fde7a468ca5ff13f695f7df42 P e358b8bf6ce346458bbbeefd27838621 [term 1 FOLLOWER]: Refusing update from remote peer 1fe6b02094dd4d8aa91c77bf1a44299e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:26:55.573037 12590 consensus_queue.cc:1035] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:26:55.573334 12593 consensus_queue.cc:1035] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:26:55.579017 12615 mvcc.cc:205] Tried to move back new op lower bound from 7316404697380765696 to 7316404695832231936. Current Snapshot: MvccSnapshot[applied={T|T < 7316404697380765696}]
I20260808 23:26:55.590176 12618 mvcc.cc:205] Tried to move back new op lower bound from 7316404697380765696 to 7316404695832231936. Current Snapshot: MvccSnapshot[applied={T|T < 7316404697380765696}]
I20260808 23:26:55.637156 11853 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:26:55.637385 11853 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:26:55.640774 11853 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280: Sending DeleteTablet for 3 replicas of tablet 8c09352fde7a468ca5ff13f695f7df42
I20260808 23:26:55.641738 12057 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c09352fde7a468ca5ff13f695f7df42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:26:55 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:48624
I20260808 23:26:55.642014 12256 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c09352fde7a468ca5ff13f695f7df42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:26:55 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:32848
I20260808 23:26:55.642782 12643 tablet_replica.cc:330] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e: stopping tablet replica
I20260808 23:26:55.642802 12461 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c09352fde7a468ca5ff13f695f7df42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:26:55 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:34976
I20260808 23:26:55.642987 12647 tablet_replica.cc:330] T 8c09352fde7a468ca5ff13f695f7df42 P e358b8bf6ce346458bbbeefd27838621: stopping tablet replica
I20260808 23:26:55.643392 12647 raft_consensus.cc:2229] T 8c09352fde7a468ca5ff13f695f7df42 P e358b8bf6ce346458bbbeefd27838621 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:26:55.643393 12643 raft_consensus.cc:2229] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:26:55.643889 12647 raft_consensus.cc:2258] T 8c09352fde7a468ca5ff13f695f7df42 P e358b8bf6ce346458bbbeefd27838621 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:26:55.644134 12643 raft_consensus.cc:2258] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:26:55.644799 11853 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49584:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:26:55.645395 11853 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:26:55.646093 12647 ts_tablet_manager.cc:1905] T 8c09352fde7a468ca5ff13f695f7df42 P e358b8bf6ce346458bbbeefd27838621: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:26:55.646287 12643 ts_tablet_manager.cc:1905] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:26:55.653180 12057 tablet_service.cc:1467] Processing CreateTablet for tablet a83c873bf6834e12a373fbf3695fc80f (DEFAULT_TABLE table=test-table [id=da9b3bd475bc4b77b7cc0040d885317d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:26:55.653589 12643 ts_tablet_manager.cc:1918] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:26:55.653640 12057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a83c873bf6834e12a373fbf3695fc80f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:55.653594 12460 tablet_service.cc:1467] Processing CreateTablet for tablet a83c873bf6834e12a373fbf3695fc80f (DEFAULT_TABLE table=test-table [id=da9b3bd475bc4b77b7cc0040d885317d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:26:55.653759 12643 log.cc:1196] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-0/wal/wals/8c09352fde7a468ca5ff13f695f7df42
I20260808 23:26:55.654098 12460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a83c873bf6834e12a373fbf3695fc80f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:55.654317 12643 ts_tablet_manager.cc:1939] T 8c09352fde7a468ca5ff13f695f7df42 P 1fe6b02094dd4d8aa91c77bf1a44299e: Deleting consensus metadata
I20260808 23:26:55.656046 11828 catalog_manager.cc:4841] TS 1fe6b02094dd4d8aa91c77bf1a44299e (127.9.44.65:39625): tablet 8c09352fde7a468ca5ff13f695f7df42 (table test-table [id=ffedd157234a4d0886a7e0eda56392a5]) successfully deleted
I20260808 23:26:55.658938 12582 tablet_bootstrap.cc:492] T a83c873bf6834e12a373fbf3695fc80f P cbe141c1a3b7472bbdca35ea0f1c849b: Bootstrap starting.
I20260808 23:26:55.659442 12256 tablet_service.cc:1467] Processing CreateTablet for tablet a83c873bf6834e12a373fbf3695fc80f (DEFAULT_TABLE table=test-table [id=da9b3bd475bc4b77b7cc0040d885317d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:26:55.659873 12256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a83c873bf6834e12a373fbf3695fc80f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:55.660272 12584 tablet_bootstrap.cc:492] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e: Bootstrap starting.
I20260808 23:26:55.660481 12582 tablet_bootstrap.cc:654] T a83c873bf6834e12a373fbf3695fc80f P cbe141c1a3b7472bbdca35ea0f1c849b: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:55.661919 12584 tablet_bootstrap.cc:654] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:55.664716 12647 ts_tablet_manager.cc:1918] T 8c09352fde7a468ca5ff13f695f7df42 P e358b8bf6ce346458bbbeefd27838621: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:26:55.664893 12647 log.cc:1196] T 8c09352fde7a468ca5ff13f695f7df42 P e358b8bf6ce346458bbbeefd27838621: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/8c09352fde7a468ca5ff13f695f7df42
I20260808 23:26:55.665421 12647 ts_tablet_manager.cc:1939] T 8c09352fde7a468ca5ff13f695f7df42 P e358b8bf6ce346458bbbeefd27838621: Deleting consensus metadata
I20260808 23:26:55.667359 11827 catalog_manager.cc:4841] TS e358b8bf6ce346458bbbeefd27838621 (127.9.44.66:41849): tablet 8c09352fde7a468ca5ff13f695f7df42 (table test-table [id=ffedd157234a4d0886a7e0eda56392a5]) successfully deleted
I20260808 23:26:55.671366 12580 tablet_bootstrap.cc:492] T a83c873bf6834e12a373fbf3695fc80f P e358b8bf6ce346458bbbeefd27838621: Bootstrap starting.
I20260808 23:26:55.672129 12584 tablet_bootstrap.cc:492] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e: No bootstrap required, opened a new log
I20260808 23:26:55.672340 12584 ts_tablet_manager.cc:1397] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.003s
I20260808 23:26:55.672955 12580 tablet_bootstrap.cc:654] T a83c873bf6834e12a373fbf3695fc80f P e358b8bf6ce346458bbbeefd27838621: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:55.673199 12584 raft_consensus.cc:348] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:55.673537 12584 raft_consensus.cc:374] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:55.673663 12584 raft_consensus.cc:729] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fe6b02094dd4d8aa91c77bf1a44299e, State: Initialized, Role: FOLLOWER
I20260808 23:26:55.674003 12584 consensus_queue.cc:260] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:55.674710 12584 ts_tablet_manager.cc:1428] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 23:26:55.684351 12648 tablet_replica.cc:330] T 8c09352fde7a468ca5ff13f695f7df42 P cbe141c1a3b7472bbdca35ea0f1c849b: stopping tablet replica
I20260808 23:26:55.684823 12648 raft_consensus.cc:2229] T 8c09352fde7a468ca5ff13f695f7df42 P cbe141c1a3b7472bbdca35ea0f1c849b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:26:55.685130 12648 raft_consensus.cc:2258] T 8c09352fde7a468ca5ff13f695f7df42 P cbe141c1a3b7472bbdca35ea0f1c849b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:26:55.687002 12648 ts_tablet_manager.cc:1905] T 8c09352fde7a468ca5ff13f695f7df42 P cbe141c1a3b7472bbdca35ea0f1c849b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:26:55.687454 12580 tablet_bootstrap.cc:492] T a83c873bf6834e12a373fbf3695fc80f P e358b8bf6ce346458bbbeefd27838621: No bootstrap required, opened a new log
I20260808 23:26:55.687626 12580 ts_tablet_manager.cc:1397] T a83c873bf6834e12a373fbf3695fc80f P e358b8bf6ce346458bbbeefd27838621: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.006s
I20260808 23:26:55.688473 12580 raft_consensus.cc:348] T a83c873bf6834e12a373fbf3695fc80f P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:55.688715 12580 raft_consensus.cc:374] T a83c873bf6834e12a373fbf3695fc80f P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:55.688819 12580 raft_consensus.cc:729] T a83c873bf6834e12a373fbf3695fc80f P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e358b8bf6ce346458bbbeefd27838621, State: Initialized, Role: FOLLOWER
I20260808 23:26:55.689041 12580 consensus_queue.cc:260] T a83c873bf6834e12a373fbf3695fc80f P e358b8bf6ce346458bbbeefd27838621 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:55.689566 12580 ts_tablet_manager.cc:1428] T a83c873bf6834e12a373fbf3695fc80f P e358b8bf6ce346458bbbeefd27838621: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 23:26:55.690537 12582 tablet_bootstrap.cc:492] T a83c873bf6834e12a373fbf3695fc80f P cbe141c1a3b7472bbdca35ea0f1c849b: No bootstrap required, opened a new log
I20260808 23:26:55.690750 12582 ts_tablet_manager.cc:1397] T a83c873bf6834e12a373fbf3695fc80f P cbe141c1a3b7472bbdca35ea0f1c849b: Time spent bootstrapping tablet: real 0.032s user 0.008s sys 0.010s
I20260808 23:26:55.691567 12582 raft_consensus.cc:348] T a83c873bf6834e12a373fbf3695fc80f P cbe141c1a3b7472bbdca35ea0f1c849b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:55.691905 12582 raft_consensus.cc:374] T a83c873bf6834e12a373fbf3695fc80f P cbe141c1a3b7472bbdca35ea0f1c849b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:55.692035 12582 raft_consensus.cc:729] T a83c873bf6834e12a373fbf3695fc80f P cbe141c1a3b7472bbdca35ea0f1c849b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbe141c1a3b7472bbdca35ea0f1c849b, State: Initialized, Role: FOLLOWER
I20260808 23:26:55.692361 12582 consensus_queue.cc:260] T a83c873bf6834e12a373fbf3695fc80f P cbe141c1a3b7472bbdca35ea0f1c849b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:55.694741 12648 ts_tablet_manager.cc:1918] T 8c09352fde7a468ca5ff13f695f7df42 P cbe141c1a3b7472bbdca35ea0f1c849b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:26:55.694828 12582 ts_tablet_manager.cc:1428] T a83c873bf6834e12a373fbf3695fc80f P cbe141c1a3b7472bbdca35ea0f1c849b: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260808 23:26:55.694934 12648 log.cc:1196] T 8c09352fde7a468ca5ff13f695f7df42 P cbe141c1a3b7472bbdca35ea0f1c849b: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/8c09352fde7a468ca5ff13f695f7df42
I20260808 23:26:55.695396 12648 ts_tablet_manager.cc:1939] T 8c09352fde7a468ca5ff13f695f7df42 P cbe141c1a3b7472bbdca35ea0f1c849b: Deleting consensus metadata
I20260808 23:26:55.697057 11837 catalog_manager.cc:4841] TS cbe141c1a3b7472bbdca35ea0f1c849b (127.9.44.67:33875): tablet 8c09352fde7a468ca5ff13f695f7df42 (table test-table [id=ffedd157234a4d0886a7e0eda56392a5]) successfully deleted
I20260808 23:26:55.819058 12642 raft_consensus.cc:482] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:26:55.819262 12642 raft_consensus.cc:504] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:55.820075 12642 leader_election.cc:288] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e358b8bf6ce346458bbbeefd27838621 (127.9.44.66:41849), cbe141c1a3b7472bbdca35ea0f1c849b (127.9.44.67:33875)
I20260808 23:26:55.820621 12289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83c873bf6834e12a373fbf3695fc80f" candidate_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e358b8bf6ce346458bbbeefd27838621" is_pre_election: true
I20260808 23:26:55.820758 12491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83c873bf6834e12a373fbf3695fc80f" candidate_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" is_pre_election: true
I20260808 23:26:55.820890 12289 raft_consensus.cc:2454] T a83c873bf6834e12a373fbf3695fc80f P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fe6b02094dd4d8aa91c77bf1a44299e in term 0.
I20260808 23:26:55.821010 12491 raft_consensus.cc:2454] T a83c873bf6834e12a373fbf3695fc80f P cbe141c1a3b7472bbdca35ea0f1c849b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fe6b02094dd4d8aa91c77bf1a44299e in term 0.
I20260808 23:26:55.821458 11991 leader_election.cc:302] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fe6b02094dd4d8aa91c77bf1a44299e, cbe141c1a3b7472bbdca35ea0f1c849b; no voters:
I20260808 23:26:55.822001 12642 raft_consensus.cc:2781] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:26:55.822149 12642 raft_consensus.cc:482] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:26:55.822255 12642 raft_consensus.cc:3037] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:55.824574 12642 raft_consensus.cc:504] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:55.825485 12289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83c873bf6834e12a373fbf3695fc80f" candidate_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e358b8bf6ce346458bbbeefd27838621"
I20260808 23:26:55.825512 12491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a83c873bf6834e12a373fbf3695fc80f" candidate_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b"
I20260808 23:26:55.825673 12491 raft_consensus.cc:3037] T a83c873bf6834e12a373fbf3695fc80f P cbe141c1a3b7472bbdca35ea0f1c849b [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:55.825695 12289 raft_consensus.cc:3037] T a83c873bf6834e12a373fbf3695fc80f P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:55.828460 12289 raft_consensus.cc:2454] T a83c873bf6834e12a373fbf3695fc80f P e358b8bf6ce346458bbbeefd27838621 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fe6b02094dd4d8aa91c77bf1a44299e in term 1.
I20260808 23:26:55.828930 11984 leader_election.cc:302] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fe6b02094dd4d8aa91c77bf1a44299e, e358b8bf6ce346458bbbeefd27838621; no voters:
I20260808 23:26:55.829253 12590 raft_consensus.cc:2781] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:26:55.829402 12590 raft_consensus.cc:686] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e [term 1 LEADER]: Becoming Leader. State: Replica: 1fe6b02094dd4d8aa91c77bf1a44299e, State: Running, Role: LEADER
I20260808 23:26:55.829460 12491 raft_consensus.cc:2454] T a83c873bf6834e12a373fbf3695fc80f P cbe141c1a3b7472bbdca35ea0f1c849b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fe6b02094dd4d8aa91c77bf1a44299e in term 1.
I20260808 23:26:55.829731 12590 consensus_queue.cc:237] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:55.829972 12642 leader_election.cc:288] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e [CANDIDATE]: Term 1 election: Requested vote from peers e358b8bf6ce346458bbbeefd27838621 (127.9.44.66:41849), cbe141c1a3b7472bbdca35ea0f1c849b (127.9.44.67:33875)
I20260808 23:26:55.832471 11853 catalog_manager.cc:5495] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fe6b02094dd4d8aa91c77bf1a44299e (127.9.44.65). New cstate: current_term: 1 leader_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } health_report { overall_health: UNKNOWN } } }
I20260808 23:26:55.879423 12491 raft_consensus.cc:1261] T a83c873bf6834e12a373fbf3695fc80f P cbe141c1a3b7472bbdca35ea0f1c849b [term 1 FOLLOWER]: Refusing update from remote peer 1fe6b02094dd4d8aa91c77bf1a44299e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:26:55.879492 12289 raft_consensus.cc:1261] T a83c873bf6834e12a373fbf3695fc80f P e358b8bf6ce346458bbbeefd27838621 [term 1 FOLLOWER]: Refusing update from remote peer 1fe6b02094dd4d8aa91c77bf1a44299e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:26:55.879966 12590 consensus_queue.cc:1035] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:26:55.880434 12593 consensus_queue.cc:1035] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:26:55.887650 12616 mvcc.cc:205] Tried to move back new op lower bound from 7316404698636574720 to 7316404698441367552. Current Snapshot: MvccSnapshot[applied={T|T < 7316404698636574720}]
W20260808 23:26:55.902313 12615 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 23:26:57.488482 11856 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:26:57.488695 11856 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:26:57.492816 11856 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280: Sending DeleteTablet for 3 replicas of tablet a83c873bf6834e12a373fbf3695fc80f
I20260808 23:26:57.493597 12057 tablet_service.cc:1514] Processing DeleteTablet for tablet a83c873bf6834e12a373fbf3695fc80f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:26:57 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:48624
I20260808 23:26:57.493721 12256 tablet_service.cc:1514] Processing DeleteTablet for tablet a83c873bf6834e12a373fbf3695fc80f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:26:57 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:32848
I20260808 23:26:57.493818 12461 tablet_service.cc:1514] Processing DeleteTablet for tablet a83c873bf6834e12a373fbf3695fc80f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:26:57 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:34976
I20260808 23:26:57.506213 12685 tablet_replica.cc:330] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e: stopping tablet replica
I20260808 23:26:57.506579 12685 raft_consensus.cc:2229] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:26:57.507124 12685 raft_consensus.cc:2258] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:26:57.508671 12688 tablet_replica.cc:330] T a83c873bf6834e12a373fbf3695fc80f P cbe141c1a3b7472bbdca35ea0f1c849b: stopping tablet replica
I20260808 23:26:57.509001 12688 raft_consensus.cc:2229] T a83c873bf6834e12a373fbf3695fc80f P cbe141c1a3b7472bbdca35ea0f1c849b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:26:57.509191 12685 ts_tablet_manager.cc:1905] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:26:57.509531 12688 raft_consensus.cc:2258] T a83c873bf6834e12a373fbf3695fc80f P cbe141c1a3b7472bbdca35ea0f1c849b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:26:57.510598 12688 ts_tablet_manager.cc:1905] T a83c873bf6834e12a373fbf3695fc80f P cbe141c1a3b7472bbdca35ea0f1c849b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:26:57.513904 12686 tablet_replica.cc:330] T a83c873bf6834e12a373fbf3695fc80f P e358b8bf6ce346458bbbeefd27838621: stopping tablet replica
I20260808 23:26:57.514155 12686 raft_consensus.cc:2229] T a83c873bf6834e12a373fbf3695fc80f P e358b8bf6ce346458bbbeefd27838621 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:26:57.514384 12686 raft_consensus.cc:2258] T a83c873bf6834e12a373fbf3695fc80f P e358b8bf6ce346458bbbeefd27838621 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:26:57.515118 12686 ts_tablet_manager.cc:1905] T a83c873bf6834e12a373fbf3695fc80f P e358b8bf6ce346458bbbeefd27838621: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 23:26:57 dist-test-slave-6bbx krb5kdc[11786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231617, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
I20260808 23:26:57.573146 12685 ts_tablet_manager.cc:1918] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 23:26:57.573146 12686 ts_tablet_manager.cc:1918] T a83c873bf6834e12a373fbf3695fc80f P e358b8bf6ce346458bbbeefd27838621: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 23:26:57.573318 12686 log.cc:1196] T a83c873bf6834e12a373fbf3695fc80f P e358b8bf6ce346458bbbeefd27838621: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/a83c873bf6834e12a373fbf3695fc80f
I20260808 23:26:57.573354 12685 log.cc:1196] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-0/wal/wals/a83c873bf6834e12a373fbf3695fc80f
I20260808 23:26:57.573371 12688 ts_tablet_manager.cc:1918] T a83c873bf6834e12a373fbf3695fc80f P cbe141c1a3b7472bbdca35ea0f1c849b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 23:26:57.573601 12688 log.cc:1196] T a83c873bf6834e12a373fbf3695fc80f P cbe141c1a3b7472bbdca35ea0f1c849b: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/a83c873bf6834e12a373fbf3695fc80f
I20260808 23:26:57.573825 12686 ts_tablet_manager.cc:1939] T a83c873bf6834e12a373fbf3695fc80f P e358b8bf6ce346458bbbeefd27838621: Deleting consensus metadata
I20260808 23:26:57.573936 12685 ts_tablet_manager.cc:1939] T a83c873bf6834e12a373fbf3695fc80f P 1fe6b02094dd4d8aa91c77bf1a44299e: Deleting consensus metadata
I20260808 23:26:57.574084 12688 ts_tablet_manager.cc:1939] T a83c873bf6834e12a373fbf3695fc80f P cbe141c1a3b7472bbdca35ea0f1c849b: Deleting consensus metadata
Aug 08 23:26:57 dist-test-slave-6bbx krb5kdc[11786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231617, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:26:57.593525 12698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:57.576983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49600 (local address 127.9.44.126:44905)
0808 23:26:57.577557 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:57.577563 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:57.577638 (+ 75us) server_negotiation.cc:407] Connection header received
0808 23:26:57.577789 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:57.577793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:57.577859 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:57.577956 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:57.579370 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:57.580370 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:57.581992 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:57.582534 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:57.582655 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:57.587809 (+ 5154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:26:57.587814 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:26:57.587818 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:26:57.587850 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:26:57.590429 (+ 2579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:57.591215 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:57.591220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:57.591222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:57.591419 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:26:57.591859 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:26:57.591864 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:26:57.591867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:26:57.592270 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:26:57.592500 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:57.592835 (+ 335us) server_negotiation.cc:299] Negotiation successful
0808 23:26:57.593050 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":273,"threads_started":1}
I20260808 23:26:57.596717 11856 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49600:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:26:57.597373 11856 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:26:57.604149 12256 tablet_service.cc:1467] Processing CreateTablet for tablet 28218731f94f4285a72b33da945f9eac (DEFAULT_TABLE table=test-table [id=b6559c2c492b47e29a1e70fe1de7a4d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:26:57.604125 12057 tablet_service.cc:1467] Processing CreateTablet for tablet 28218731f94f4285a72b33da945f9eac (DEFAULT_TABLE table=test-table [id=b6559c2c492b47e29a1e70fe1de7a4d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:26:57.604578 12057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28218731f94f4285a72b33da945f9eac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:57.604600 12256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28218731f94f4285a72b33da945f9eac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:57.604777 12461 tablet_service.cc:1467] Processing CreateTablet for tablet 28218731f94f4285a72b33da945f9eac (DEFAULT_TABLE table=test-table [id=b6559c2c492b47e29a1e70fe1de7a4d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:26:57.605178 12461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28218731f94f4285a72b33da945f9eac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:57.611471 12700 tablet_bootstrap.cc:492] T 28218731f94f4285a72b33da945f9eac P 1fe6b02094dd4d8aa91c77bf1a44299e: Bootstrap starting.
I20260808 23:26:57.612116 12701 tablet_bootstrap.cc:492] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621: Bootstrap starting.
I20260808 23:26:57.613545 12700 tablet_bootstrap.cc:654] T 28218731f94f4285a72b33da945f9eac P 1fe6b02094dd4d8aa91c77bf1a44299e: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:57.614318 12701 tablet_bootstrap.cc:654] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:57.617064 12702 tablet_bootstrap.cc:492] T 28218731f94f4285a72b33da945f9eac P cbe141c1a3b7472bbdca35ea0f1c849b: Bootstrap starting.
I20260808 23:26:57.617765 12701 tablet_bootstrap.cc:492] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621: No bootstrap required, opened a new log
I20260808 23:26:57.617993 12701 ts_tablet_manager.cc:1397] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 23:26:57.619257 12701 raft_consensus.cc:348] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:57.619606 12701 raft_consensus.cc:374] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:57.619798 12701 raft_consensus.cc:729] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e358b8bf6ce346458bbbeefd27838621, State: Initialized, Role: FOLLOWER
I20260808 23:26:57.619951 12702 tablet_bootstrap.cc:654] T 28218731f94f4285a72b33da945f9eac P cbe141c1a3b7472bbdca35ea0f1c849b: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:57.620170 12701 consensus_queue.cc:260] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:57.620688 12700 tablet_bootstrap.cc:492] T 28218731f94f4285a72b33da945f9eac P 1fe6b02094dd4d8aa91c77bf1a44299e: No bootstrap required, opened a new log
I20260808 23:26:57.620920 12700 ts_tablet_manager.cc:1397] T 28218731f94f4285a72b33da945f9eac P 1fe6b02094dd4d8aa91c77bf1a44299e: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260808 23:26:57.621182 12701 ts_tablet_manager.cc:1428] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 23:26:57.622181 12700 raft_consensus.cc:348] T 28218731f94f4285a72b33da945f9eac P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:57.622625 12700 raft_consensus.cc:374] T 28218731f94f4285a72b33da945f9eac P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:57.622805 12702 tablet_bootstrap.cc:492] T 28218731f94f4285a72b33da945f9eac P cbe141c1a3b7472bbdca35ea0f1c849b: No bootstrap required, opened a new log
I20260808 23:26:57.622851 12700 raft_consensus.cc:729] T 28218731f94f4285a72b33da945f9eac P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fe6b02094dd4d8aa91c77bf1a44299e, State: Initialized, Role: FOLLOWER
I20260808 23:26:57.622941 12702 ts_tablet_manager.cc:1397] T 28218731f94f4285a72b33da945f9eac P cbe141c1a3b7472bbdca35ea0f1c849b: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260808 23:26:57.623129 12700 consensus_queue.cc:260] T 28218731f94f4285a72b33da945f9eac P 1fe6b02094dd4d8aa91c77bf1a44299e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:57.623705 12702 raft_consensus.cc:348] T 28218731f94f4285a72b33da945f9eac P cbe141c1a3b7472bbdca35ea0f1c849b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:57.623994 12700 ts_tablet_manager.cc:1428] T 28218731f94f4285a72b33da945f9eac P 1fe6b02094dd4d8aa91c77bf1a44299e: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260808 23:26:57.624104 12702 raft_consensus.cc:374] T 28218731f94f4285a72b33da945f9eac P cbe141c1a3b7472bbdca35ea0f1c849b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:57.624300 12702 raft_consensus.cc:729] T 28218731f94f4285a72b33da945f9eac P cbe141c1a3b7472bbdca35ea0f1c849b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbe141c1a3b7472bbdca35ea0f1c849b, State: Initialized, Role: FOLLOWER
I20260808 23:26:57.624610 12702 consensus_queue.cc:260] T 28218731f94f4285a72b33da945f9eac P cbe141c1a3b7472bbdca35ea0f1c849b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:57.625775 12702 ts_tablet_manager.cc:1428] T 28218731f94f4285a72b33da945f9eac P cbe141c1a3b7472bbdca35ea0f1c849b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 23:26:57.648036 11827 catalog_manager.cc:4841] TS e358b8bf6ce346458bbbeefd27838621 (127.9.44.66:41849): tablet a83c873bf6834e12a373fbf3695fc80f (table test-table [id=da9b3bd475bc4b77b7cc0040d885317d]) successfully deleted
I20260808 23:26:57.648069 11837 catalog_manager.cc:4841] TS cbe141c1a3b7472bbdca35ea0f1c849b (127.9.44.67:33875): tablet a83c873bf6834e12a373fbf3695fc80f (table test-table [id=da9b3bd475bc4b77b7cc0040d885317d]) successfully deleted
I20260808 23:26:57.648134 11828 catalog_manager.cc:4841] TS 1fe6b02094dd4d8aa91c77bf1a44299e (127.9.44.65:39625): tablet a83c873bf6834e12a373fbf3695fc80f (table test-table [id=da9b3bd475bc4b77b7cc0040d885317d]) successfully deleted
I20260808 23:26:57.702412 12705 raft_consensus.cc:482] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:26:57.702683 12705 raft_consensus.cc:504] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:57.704360 12705 leader_election.cc:288] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fe6b02094dd4d8aa91c77bf1a44299e (127.9.44.65:39625), cbe141c1a3b7472bbdca35ea0f1c849b (127.9.44.67:33875)
I20260808 23:26:57.721282 12713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:57.704723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:32953 (local address 127.9.44.67:33875)
0808 23:26:57.715835 (+ 11112us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:57.715842 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:57.715906 (+ 64us) server_negotiation.cc:407] Connection header received
0808 23:26:57.715989 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:57.715995 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:57.716076 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:57.716227 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:26:57.716878 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:57.717826 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:57.719496 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:57.720334 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:57.720448 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:57.720587 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:57.720841 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 23:26:57.720966 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10833,"thread_start_us":10589,"threads_started":1}
I20260808 23:26:57.721308 12715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:57.704737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:33875 (local address 127.9.44.66:32953)
0808 23:26:57.715145 (+ 10408us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:57.715168 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:57.715432 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:57.716354 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:57.716359 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:57.716386 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:26:57.716692 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:57.716702 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:57.717962 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:57.717966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:57.719358 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:57.719371 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:57.720530 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:57.720537 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:57.720696 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:57.720721 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 23:26:57.720802 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 23:26:57.720919 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10281,"thread_start_us":10124,"threads_started":1}
I20260808 23:26:57.721956 12491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28218731f94f4285a72b33da945f9eac" candidate_uuid: "e358b8bf6ce346458bbbeefd27838621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" is_pre_election: true
I20260808 23:26:57.722187 12711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:57.704655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:45823 (local address 127.9.44.65:39625)
0808 23:26:57.716006 (+ 11351us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:57.716012 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:57.716559 (+ 547us) server_negotiation.cc:407] Connection header received
0808 23:26:57.716744 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:57.716748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:57.716808 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:57.716909 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:26:57.717471 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:57.718207 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:57.720104 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:57.720923 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:57.721038 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:57.721228 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:57.721646 (+ 418us) server_negotiation.cc:299] Negotiation successful
0808 23:26:57.721795 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11202,"thread_start_us":11030,"threads_started":1}
I20260808 23:26:57.722221 12718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:57.704733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:39625 (local address 127.9.44.66:45823)
0808 23:26:57.716448 (+ 11715us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:57.716468 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:57.716608 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:57.716931 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:57.716935 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:57.716961 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:26:57.717284 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:57.717295 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:57.718354 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:57.718358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:57.719950 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:57.719964 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:57.721217 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:57.721224 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:57.721480 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:57.721513 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 23:26:57.721602 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 23:26:57.721717 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11567,"mutex_wait_us":28,"thread_start_us":11262,"threads_started":1}
I20260808 23:26:57.722250 12491 raft_consensus.cc:2454] T 28218731f94f4285a72b33da945f9eac P cbe141c1a3b7472bbdca35ea0f1c849b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e358b8bf6ce346458bbbeefd27838621 in term 0.
I20260808 23:26:57.722734 12194 leader_election.cc:302] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbe141c1a3b7472bbdca35ea0f1c849b, e358b8bf6ce346458bbbeefd27838621; no voters:
I20260808 23:26:57.723119 12705 raft_consensus.cc:2781] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:26:57.723234 12705 raft_consensus.cc:482] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:26:57.723321 12705 raft_consensus.cc:3037] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:57.723311 12087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28218731f94f4285a72b33da945f9eac" candidate_uuid: "e358b8bf6ce346458bbbeefd27838621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" is_pre_election: true
I20260808 23:26:57.723862 12087 raft_consensus.cc:2454] T 28218731f94f4285a72b33da945f9eac P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e358b8bf6ce346458bbbeefd27838621 in term 0.
I20260808 23:26:57.768260 12705 raft_consensus.cc:504] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:57.769632 12087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28218731f94f4285a72b33da945f9eac" candidate_uuid: "e358b8bf6ce346458bbbeefd27838621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e"
I20260808 23:26:57.769771 12491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28218731f94f4285a72b33da945f9eac" candidate_uuid: "e358b8bf6ce346458bbbeefd27838621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b"
I20260808 23:26:57.769879 12087 raft_consensus.cc:3037] T 28218731f94f4285a72b33da945f9eac P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:57.769987 12491 raft_consensus.cc:3037] T 28218731f94f4285a72b33da945f9eac P cbe141c1a3b7472bbdca35ea0f1c849b [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:57.772971 12087 raft_consensus.cc:2454] T 28218731f94f4285a72b33da945f9eac P 1fe6b02094dd4d8aa91c77bf1a44299e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e358b8bf6ce346458bbbeefd27838621 in term 1.
I20260808 23:26:57.773550 12705 leader_election.cc:288] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621 [CANDIDATE]: Term 1 election: Requested vote from peers 1fe6b02094dd4d8aa91c77bf1a44299e (127.9.44.65:39625), cbe141c1a3b7472bbdca35ea0f1c849b (127.9.44.67:33875)
I20260808 23:26:57.773592 12190 leader_election.cc:302] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fe6b02094dd4d8aa91c77bf1a44299e, e358b8bf6ce346458bbbeefd27838621; no voters:
I20260808 23:26:57.774127 12705 raft_consensus.cc:2781] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:26:57.774792 12705 raft_consensus.cc:686] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621 [term 1 LEADER]: Becoming Leader. State: Replica: e358b8bf6ce346458bbbeefd27838621, State: Running, Role: LEADER
I20260808 23:26:57.775401 12705 consensus_queue.cc:237] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:57.779201 11856 catalog_manager.cc:5495] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621 reported cstate change: term changed from 0 to 1, leader changed from <none> to e358b8bf6ce346458bbbeefd27838621 (127.9.44.66). New cstate: current_term: 1 leader_uuid: "e358b8bf6ce346458bbbeefd27838621" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } health_report { overall_health: UNKNOWN } } }
I20260808 23:26:57.818115 12725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:57.805403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59280 (local address 127.9.44.66:41849)
0808 23:26:57.812109 (+ 6706us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:57.812113 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:57.812132 (+ 19us) server_negotiation.cc:407] Connection header received
0808 23:26:57.812210 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:57.812214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:57.812299 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:57.812447 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:26:57.812925 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:57.813716 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:57.815752 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:57.816211 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:57.816302 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:57.816916 (+ 614us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:26:57.817483 (+ 567us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:26:57.817586 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:57.817735 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 23:26:57.817801 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6609,"thread_start_us":6480,"threads_started":1}
I20260808 23:26:57.833981 12711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:57.828533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:52803 (local address 127.9.44.65:39625)
0808 23:26:57.828722 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:57.828726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:57.829707 (+ 981us) server_negotiation.cc:407] Connection header received
0808 23:26:57.829841 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:57.829845 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:57.829910 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:57.829982 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:26:57.830453 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:57.831240 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:57.832606 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:57.833044 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:57.833136 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:57.833475 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:26:57.833567 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:26:57.833627 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:57.833713 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 23:26:57.833770 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 23:26:57.834052 12728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:57.828682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:39625 (local address 127.9.44.66:52803)
0808 23:26:57.829604 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:57.829619 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:57.829742 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:57.829994 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:57.829997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:57.830017 (+ 20us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 23:26:57.830339 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:57.830348 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:57.831336 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:57.831340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:57.832482 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:57.832495 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:57.833170 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:57.833174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:57.833286 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:57.833341 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 23:26:57.833629 (+ 288us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 23:26:57.833636 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 23:26:57.833689 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 23:26:57.833786 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":750,"thread_start_us":518,"threads_started":1}
I20260808 23:26:57.842270 12087 raft_consensus.cc:1261] T 28218731f94f4285a72b33da945f9eac P 1fe6b02094dd4d8aa91c77bf1a44299e [term 1 FOLLOWER]: Refusing update from remote peer e358b8bf6ce346458bbbeefd27838621: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:26:57.842989 12721 consensus_queue.cc:1035] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:26:57.869870 12491 raft_consensus.cc:2454] T 28218731f94f4285a72b33da945f9eac P cbe141c1a3b7472bbdca35ea0f1c849b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e358b8bf6ce346458bbbeefd27838621 in term 1.
I20260808 23:26:57.870504 12492 raft_consensus.cc:1261] T 28218731f94f4285a72b33da945f9eac P cbe141c1a3b7472bbdca35ea0f1c849b [term 1 FOLLOWER]: Refusing update from remote peer e358b8bf6ce346458bbbeefd27838621: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:26:57.870860 12725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:57.862145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:57877 (local address 127.9.44.66:41849)
0808 23:26:57.862312 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:57.862318 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:57.863590 (+ 1272us) server_negotiation.cc:407] Connection header received
0808 23:26:57.863826 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:57.863832 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:57.863901 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:57.863970 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:26:57.865166 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:57.866112 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:57.868603 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:57.869151 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:57.869266 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:57.870020 (+ 754us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:26:57.870137 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:26:57.870197 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:57.870408 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 23:26:57.870482 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 23:26:57.871654 12705 consensus_queue.cc:1035] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:26:57.871830 12731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:57.862188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:41849 (local address 127.9.44.65:57877)
0808 23:26:57.863470 (+ 1282us) negotiation.cc:107] Waiting for socket to connect
0808 23:26:57.863506 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 23:26:57.863703 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:26:57.864011 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:26:57.864016 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:26:57.864028 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 23:26:57.864930 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:57.864947 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:57.866281 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:57.866286 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:57.868471 (+ 2185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:26:57.868485 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:26:57.869433 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:26:57.869444 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:26:57.869756 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:57.869806 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 23:26:57.870289 (+ 483us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 23:26:57.870299 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 23:26:57.871104 (+ 805us) client_negotiation.cc:240] Negotiation successful
0808 23:26:57.871251 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1080,"thread_start_us":805,"threads_started":1}
I20260808 23:26:57.911242 11856 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:26:57.911595 11856 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:26:57.914875 11856 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280: Sending DeleteTablet for 3 replicas of tablet 28218731f94f4285a72b33da945f9eac
I20260808 23:26:57.915789 12057 tablet_service.cc:1514] Processing DeleteTablet for tablet 28218731f94f4285a72b33da945f9eac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:26:57 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:48624
I20260808 23:26:57.915972 12256 tablet_service.cc:1514] Processing DeleteTablet for tablet 28218731f94f4285a72b33da945f9eac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:26:57 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:32848
I20260808 23:26:57.916118 12685 tablet_replica.cc:330] T 28218731f94f4285a72b33da945f9eac P 1fe6b02094dd4d8aa91c77bf1a44299e: stopping tablet replica
I20260808 23:26:57.916388 12686 tablet_replica.cc:330] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621: stopping tablet replica
I20260808 23:26:57.916380 12685 raft_consensus.cc:2229] T 28218731f94f4285a72b33da945f9eac P 1fe6b02094dd4d8aa91c77bf1a44299e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:26:57.916455 12461 tablet_service.cc:1514] Processing DeleteTablet for tablet 28218731f94f4285a72b33da945f9eac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:26:57 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:34976
I20260808 23:26:57.916586 12685 raft_consensus.cc:2258] T 28218731f94f4285a72b33da945f9eac P 1fe6b02094dd4d8aa91c77bf1a44299e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:26:57.916685 12686 raft_consensus.cc:2229] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:26:57.916817 12688 tablet_replica.cc:330] T 28218731f94f4285a72b33da945f9eac P cbe141c1a3b7472bbdca35ea0f1c849b: stopping tablet replica
I20260808 23:26:57.917047 12688 raft_consensus.cc:2229] T 28218731f94f4285a72b33da945f9eac P cbe141c1a3b7472bbdca35ea0f1c849b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:26:57.917250 12688 raft_consensus.cc:2258] T 28218731f94f4285a72b33da945f9eac P cbe141c1a3b7472bbdca35ea0f1c849b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:26:57.917404 12686 raft_consensus.cc:2258] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:26:57.918330 12685 ts_tablet_manager.cc:1905] T 28218731f94f4285a72b33da945f9eac P 1fe6b02094dd4d8aa91c77bf1a44299e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:26:57.918330 12688 ts_tablet_manager.cc:1905] T 28218731f94f4285a72b33da945f9eac P cbe141c1a3b7472bbdca35ea0f1c849b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:26:57.918635 12686 ts_tablet_manager.cc:1905] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:26:57.923334 11856 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49600:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:26:57.923942 11856 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:26:57.930167 12057 tablet_service.cc:1467] Processing CreateTablet for tablet 2142ca229e73471fb4b60df58df1ecb5 (DEFAULT_TABLE table=test-table [id=98cbb8af6d2f45a0aea66bb5c542f188]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:26:57.930554 12256 tablet_service.cc:1467] Processing CreateTablet for tablet 2142ca229e73471fb4b60df58df1ecb5 (DEFAULT_TABLE table=test-table [id=98cbb8af6d2f45a0aea66bb5c542f188]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:26:57.931093 12461 tablet_service.cc:1467] Processing CreateTablet for tablet 2142ca229e73471fb4b60df58df1ecb5 (DEFAULT_TABLE table=test-table [id=98cbb8af6d2f45a0aea66bb5c542f188]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:26:57.970968 12256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2142ca229e73471fb4b60df58df1ecb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:57.971207 12461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2142ca229e73471fb4b60df58df1ecb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:57.974704 12057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2142ca229e73471fb4b60df58df1ecb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:26:57.984544 12701 tablet_bootstrap.cc:492] T 2142ca229e73471fb4b60df58df1ecb5 P e358b8bf6ce346458bbbeefd27838621: Bootstrap starting.
I20260808 23:26:57.986094 12700 tablet_bootstrap.cc:492] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e: Bootstrap starting.
I20260808 23:26:57.986579 12701 tablet_bootstrap.cc:654] T 2142ca229e73471fb4b60df58df1ecb5 P e358b8bf6ce346458bbbeefd27838621: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:57.987705 12700 tablet_bootstrap.cc:654] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:57.993412 12700 tablet_bootstrap.cc:492] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e: No bootstrap required, opened a new log
I20260808 23:26:57.993788 12700 ts_tablet_manager.cc:1397] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.002s
I20260808 23:26:57.994143 12701 tablet_bootstrap.cc:492] T 2142ca229e73471fb4b60df58df1ecb5 P e358b8bf6ce346458bbbeefd27838621: No bootstrap required, opened a new log
I20260808 23:26:57.994730 12702 tablet_bootstrap.cc:492] T 2142ca229e73471fb4b60df58df1ecb5 P cbe141c1a3b7472bbdca35ea0f1c849b: Bootstrap starting.
I20260808 23:26:57.995360 12701 ts_tablet_manager.cc:1397] T 2142ca229e73471fb4b60df58df1ecb5 P e358b8bf6ce346458bbbeefd27838621: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260808 23:26:57.995218 12700 raft_consensus.cc:348] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:57.995841 12700 raft_consensus.cc:374] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:57.996101 12700 raft_consensus.cc:729] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fe6b02094dd4d8aa91c77bf1a44299e, State: Initialized, Role: FOLLOWER
I20260808 23:26:57.996474 12700 consensus_queue.cc:260] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:57.996655 12702 tablet_bootstrap.cc:654] T 2142ca229e73471fb4b60df58df1ecb5 P cbe141c1a3b7472bbdca35ea0f1c849b: Neither blocks nor log segments found. Creating new log.
I20260808 23:26:57.996599 12701 raft_consensus.cc:348] T 2142ca229e73471fb4b60df58df1ecb5 P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:57.997037 12701 raft_consensus.cc:374] T 2142ca229e73471fb4b60df58df1ecb5 P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:57.997215 12701 raft_consensus.cc:729] T 2142ca229e73471fb4b60df58df1ecb5 P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e358b8bf6ce346458bbbeefd27838621, State: Initialized, Role: FOLLOWER
I20260808 23:26:57.997823 12700 ts_tablet_manager.cc:1428] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 23:26:58.001706 12701 consensus_queue.cc:260] T 2142ca229e73471fb4b60df58df1ecb5 P e358b8bf6ce346458bbbeefd27838621 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:58.002604 12701 ts_tablet_manager.cc:1428] T 2142ca229e73471fb4b60df58df1ecb5 P e358b8bf6ce346458bbbeefd27838621: Time spent starting tablet: real 0.007s user 0.000s sys 0.003s
I20260808 23:26:58.003298 12702 tablet_bootstrap.cc:492] T 2142ca229e73471fb4b60df58df1ecb5 P cbe141c1a3b7472bbdca35ea0f1c849b: No bootstrap required, opened a new log
I20260808 23:26:58.003643 12702 ts_tablet_manager.cc:1397] T 2142ca229e73471fb4b60df58df1ecb5 P cbe141c1a3b7472bbdca35ea0f1c849b: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260808 23:26:58.004910 12702 raft_consensus.cc:348] T 2142ca229e73471fb4b60df58df1ecb5 P cbe141c1a3b7472bbdca35ea0f1c849b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:58.005220 12702 raft_consensus.cc:374] T 2142ca229e73471fb4b60df58df1ecb5 P cbe141c1a3b7472bbdca35ea0f1c849b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:26:58.005403 12702 raft_consensus.cc:729] T 2142ca229e73471fb4b60df58df1ecb5 P cbe141c1a3b7472bbdca35ea0f1c849b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbe141c1a3b7472bbdca35ea0f1c849b, State: Initialized, Role: FOLLOWER
I20260808 23:26:58.005861 12702 consensus_queue.cc:260] T 2142ca229e73471fb4b60df58df1ecb5 P cbe141c1a3b7472bbdca35ea0f1c849b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:58.006790 12702 ts_tablet_manager.cc:1428] T 2142ca229e73471fb4b60df58df1ecb5 P cbe141c1a3b7472bbdca35ea0f1c849b: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260808 23:26:58.071151 12688 ts_tablet_manager.cc:1918] T 28218731f94f4285a72b33da945f9eac P cbe141c1a3b7472bbdca35ea0f1c849b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:26:58.071144 12686 ts_tablet_manager.cc:1918] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:26:58.071468 12688 log.cc:1196] T 28218731f94f4285a72b33da945f9eac P cbe141c1a3b7472bbdca35ea0f1c849b: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/28218731f94f4285a72b33da945f9eac
I20260808 23:26:58.071617 12686 log.cc:1196] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/28218731f94f4285a72b33da945f9eac
I20260808 23:26:58.072208 12688 ts_tablet_manager.cc:1939] T 28218731f94f4285a72b33da945f9eac P cbe141c1a3b7472bbdca35ea0f1c849b: Deleting consensus metadata
I20260808 23:26:58.072582 12686 ts_tablet_manager.cc:1939] T 28218731f94f4285a72b33da945f9eac P e358b8bf6ce346458bbbeefd27838621: Deleting consensus metadata
I20260808 23:26:58.073669 12685 ts_tablet_manager.cc:1918] T 28218731f94f4285a72b33da945f9eac P 1fe6b02094dd4d8aa91c77bf1a44299e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:26:58.073880 11837 catalog_manager.cc:4841] TS cbe141c1a3b7472bbdca35ea0f1c849b (127.9.44.67:33875): tablet 28218731f94f4285a72b33da945f9eac (table test-table [id=b6559c2c492b47e29a1e70fe1de7a4d0]) successfully deleted
I20260808 23:26:58.074026 12685 log.cc:1196] T 28218731f94f4285a72b33da945f9eac P 1fe6b02094dd4d8aa91c77bf1a44299e: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-0/wal/wals/28218731f94f4285a72b33da945f9eac
I20260808 23:26:58.074373 11827 catalog_manager.cc:4841] TS e358b8bf6ce346458bbbeefd27838621 (127.9.44.66:41849): tablet 28218731f94f4285a72b33da945f9eac (table test-table [id=b6559c2c492b47e29a1e70fe1de7a4d0]) successfully deleted
I20260808 23:26:58.074779 12685 ts_tablet_manager.cc:1939] T 28218731f94f4285a72b33da945f9eac P 1fe6b02094dd4d8aa91c77bf1a44299e: Deleting consensus metadata
I20260808 23:26:58.133985 11828 catalog_manager.cc:4841] TS 1fe6b02094dd4d8aa91c77bf1a44299e (127.9.44.65:39625): tablet 28218731f94f4285a72b33da945f9eac (table test-table [id=b6559c2c492b47e29a1e70fe1de7a4d0]) successfully deleted
I20260808 23:26:58.163358 12659 raft_consensus.cc:482] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:26:58.163726 12659 raft_consensus.cc:504] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:58.165055 12659 leader_election.cc:288] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e358b8bf6ce346458bbbeefd27838621 (127.9.44.66:41849), cbe141c1a3b7472bbdca35ea0f1c849b (127.9.44.67:33875)
I20260808 23:26:58.165690 12492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2142ca229e73471fb4b60df58df1ecb5" candidate_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" is_pre_election: true
I20260808 23:26:58.165891 12285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2142ca229e73471fb4b60df58df1ecb5" candidate_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e358b8bf6ce346458bbbeefd27838621" is_pre_election: true
I20260808 23:26:58.166260 12492 raft_consensus.cc:2454] T 2142ca229e73471fb4b60df58df1ecb5 P cbe141c1a3b7472bbdca35ea0f1c849b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fe6b02094dd4d8aa91c77bf1a44299e in term 0.
I20260808 23:26:58.166415 12285 raft_consensus.cc:2454] T 2142ca229e73471fb4b60df58df1ecb5 P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fe6b02094dd4d8aa91c77bf1a44299e in term 0.
I20260808 23:26:58.166817 11991 leader_election.cc:302] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fe6b02094dd4d8aa91c77bf1a44299e, cbe141c1a3b7472bbdca35ea0f1c849b; no voters:
I20260808 23:26:58.167176 12659 raft_consensus.cc:2781] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:26:58.167358 12659 raft_consensus.cc:482] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:26:58.167479 12659 raft_consensus.cc:3037] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:58.172849 12659 raft_consensus.cc:504] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:58.173629 12659 leader_election.cc:288] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e [CANDIDATE]: Term 1 election: Requested vote from peers e358b8bf6ce346458bbbeefd27838621 (127.9.44.66:41849), cbe141c1a3b7472bbdca35ea0f1c849b (127.9.44.67:33875)
I20260808 23:26:58.174098 12285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2142ca229e73471fb4b60df58df1ecb5" candidate_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e358b8bf6ce346458bbbeefd27838621"
I20260808 23:26:58.174371 12285 raft_consensus.cc:3037] T 2142ca229e73471fb4b60df58df1ecb5 P e358b8bf6ce346458bbbeefd27838621 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:58.174661 12492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2142ca229e73471fb4b60df58df1ecb5" candidate_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b"
I20260808 23:26:58.175031 12492 raft_consensus.cc:3037] T 2142ca229e73471fb4b60df58df1ecb5 P cbe141c1a3b7472bbdca35ea0f1c849b [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:26:58.274369 12492 raft_consensus.cc:2454] T 2142ca229e73471fb4b60df58df1ecb5 P cbe141c1a3b7472bbdca35ea0f1c849b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fe6b02094dd4d8aa91c77bf1a44299e in term 1.
I20260808 23:26:58.274646 12285 raft_consensus.cc:2454] T 2142ca229e73471fb4b60df58df1ecb5 P e358b8bf6ce346458bbbeefd27838621 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fe6b02094dd4d8aa91c77bf1a44299e in term 1.
I20260808 23:26:58.275069 11991 leader_election.cc:302] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fe6b02094dd4d8aa91c77bf1a44299e, cbe141c1a3b7472bbdca35ea0f1c849b; no voters:
I20260808 23:26:58.275653 12659 raft_consensus.cc:2781] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:26:58.276062 12659 raft_consensus.cc:686] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 1 LEADER]: Becoming Leader. State: Replica: 1fe6b02094dd4d8aa91c77bf1a44299e, State: Running, Role: LEADER
I20260808 23:26:58.276528 12659 consensus_queue.cc:237] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } }
I20260808 23:26:58.280460 11853 catalog_manager.cc:5495] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fe6b02094dd4d8aa91c77bf1a44299e (127.9.44.65). New cstate: current_term: 1 leader_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fe6b02094dd4d8aa91c77bf1a44299e" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 } health_report { overall_health: UNKNOWN } } }
I20260808 23:26:58.336485 12711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:58.323618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47018 (local address 127.9.44.65:39625)
0808 23:26:58.323806 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:58.323812 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:58.328901 (+ 5089us) server_negotiation.cc:407] Connection header received
0808 23:26:58.329070 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:58.329075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:58.329147 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:58.329228 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:26:58.330225 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:58.331700 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:58.333882 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:26:58.334740 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:26:58.334903 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:26:58.335499 (+ 596us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:26:58.335655 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:26:58.335770 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 23:26:58.335946 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 23:26:58.336022 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 23:26:58.348224 12285 raft_consensus.cc:1261] T 2142ca229e73471fb4b60df58df1ecb5 P e358b8bf6ce346458bbbeefd27838621 [term 1 FOLLOWER]: Refusing update from remote peer 1fe6b02094dd4d8aa91c77bf1a44299e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:26:58.348255 12492 raft_consensus.cc:1261] T 2142ca229e73471fb4b60df58df1ecb5 P cbe141c1a3b7472bbdca35ea0f1c849b [term 1 FOLLOWER]: Refusing update from remote peer 1fe6b02094dd4d8aa91c77bf1a44299e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:26:58.348917 12659 consensus_queue.cc:1035] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e358b8bf6ce346458bbbeefd27838621" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 41849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:26:58.349336 12676 consensus_queue.cc:1035] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbe141c1a3b7472bbdca35ea0f1c849b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 23:26:58.372910 12660 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 23:26:59.968227 11856 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:26:59.968549 11856 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:26:59.974622 11856 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed4ee6cf29134b0889de70142db69280: Sending DeleteTablet for 3 replicas of tablet 2142ca229e73471fb4b60df58df1ecb5
I20260808 23:26:59.975700 12256 tablet_service.cc:1514] Processing DeleteTablet for tablet 2142ca229e73471fb4b60df58df1ecb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:26:59 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:32848
I20260808 23:26:59.975768 12057 tablet_service.cc:1514] Processing DeleteTablet for tablet 2142ca229e73471fb4b60df58df1ecb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:26:59 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:48624
I20260808 23:26:59.975783 12461 tablet_service.cc:1514] Processing DeleteTablet for tablet 2142ca229e73471fb4b60df58df1ecb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:26:59 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:34976
I20260808 23:26:59.976374 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 11957
I20260808 23:26:59.988644 12780 tablet_replica.cc:330] T 2142ca229e73471fb4b60df58df1ecb5 P e358b8bf6ce346458bbbeefd27838621: stopping tablet replica
I20260808 23:26:59.988895 12780 raft_consensus.cc:2229] T 2142ca229e73471fb4b60df58df1ecb5 P e358b8bf6ce346458bbbeefd27838621 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:26:59.989396 12780 raft_consensus.cc:2258] T 2142ca229e73471fb4b60df58df1ecb5 P e358b8bf6ce346458bbbeefd27838621 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:26:59.990051 12781 tablet_replica.cc:330] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e: stopping tablet replica
I20260808 23:26:59.990314 12781 raft_consensus.cc:2229] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:26:59.990478 12782 tablet_replica.cc:330] T 2142ca229e73471fb4b60df58df1ecb5 P cbe141c1a3b7472bbdca35ea0f1c849b: stopping tablet replica
I20260808 23:26:59.990552 12780 ts_tablet_manager.cc:1905] T 2142ca229e73471fb4b60df58df1ecb5 P e358b8bf6ce346458bbbeefd27838621: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:26:59.990749 12781 raft_consensus.cc:2258] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:26:59.990933 12782 raft_consensus.cc:2229] T 2142ca229e73471fb4b60df58df1ecb5 P cbe141c1a3b7472bbdca35ea0f1c849b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:26:59.991449 12782 raft_consensus.cc:2258] T 2142ca229e73471fb4b60df58df1ecb5 P cbe141c1a3b7472bbdca35ea0f1c849b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:26:59.991823 12781 ts_tablet_manager.cc:1905] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:26:59.992982 12782 ts_tablet_manager.cc:1905] T 2142ca229e73471fb4b60df58df1ecb5 P cbe141c1a3b7472bbdca35ea0f1c849b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:27:00.000232 12780 ts_tablet_manager.cc:1918] T 2142ca229e73471fb4b60df58df1ecb5 P e358b8bf6ce346458bbbeefd27838621: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 23:27:00.000517 12780 log.cc:1196] T 2142ca229e73471fb4b60df58df1ecb5 P e358b8bf6ce346458bbbeefd27838621: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/2142ca229e73471fb4b60df58df1ecb5
I20260808 23:27:00.000799 12781 ts_tablet_manager.cc:1918] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 23:27:00.000981 12781 log.cc:1196] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-0/wal/wals/2142ca229e73471fb4b60df58df1ecb5
I20260808 23:27:00.001319 12780 ts_tablet_manager.cc:1939] T 2142ca229e73471fb4b60df58df1ecb5 P e358b8bf6ce346458bbbeefd27838621: Deleting consensus metadata
I20260808 23:27:00.001781 12781 ts_tablet_manager.cc:1939] T 2142ca229e73471fb4b60df58df1ecb5 P 1fe6b02094dd4d8aa91c77bf1a44299e: Deleting consensus metadata
I20260808 23:27:00.001787 12782 ts_tablet_manager.cc:1918] T 2142ca229e73471fb4b60df58df1ecb5 P cbe141c1a3b7472bbdca35ea0f1c849b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 23:27:00.001979 12782 log.cc:1196] T 2142ca229e73471fb4b60df58df1ecb5 P cbe141c1a3b7472bbdca35ea0f1c849b: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/2142ca229e73471fb4b60df58df1ecb5
I20260808 23:27:00.002550 11827 catalog_manager.cc:4841] TS e358b8bf6ce346458bbbeefd27838621 (127.9.44.66:41849): tablet 2142ca229e73471fb4b60df58df1ecb5 (table test-table [id=98cbb8af6d2f45a0aea66bb5c542f188]) successfully deleted
I20260808 23:27:00.002785 12782 ts_tablet_manager.cc:1939] T 2142ca229e73471fb4b60df58df1ecb5 P cbe141c1a3b7472bbdca35ea0f1c849b: Deleting consensus metadata
I20260808 23:27:00.003552 11828 catalog_manager.cc:4841] TS 1fe6b02094dd4d8aa91c77bf1a44299e (127.9.44.65:39625): tablet 2142ca229e73471fb4b60df58df1ecb5 (table test-table [id=98cbb8af6d2f45a0aea66bb5c542f188]) successfully deleted
I20260808 23:27:00.003912 11837 catalog_manager.cc:4841] TS cbe141c1a3b7472bbdca35ea0f1c849b (127.9.44.67:33875): tablet 2142ca229e73471fb4b60df58df1ecb5 (table test-table [id=98cbb8af6d2f45a0aea66bb5c542f188]) successfully deleted
Aug 08 23:27:00 dist-test-slave-6bbx krb5kdc[11786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231617, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.44.65@KRBTEST.COM
I20260808 23:27:00.023979 12784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:26:59.977742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47020 (local address 127.9.44.65:39625)
0808 23:26:59.996288 (+ 18546us) server_negotiation.cc:206] Beginning negotiation
0808 23:26:59.996294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:26:59.996342 (+ 48us) server_negotiation.cc:407] Connection header received
0808 23:26:59.996402 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:26:59.996407 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:26:59.996481 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:26:59.999190 (+ 2709us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:26:59.999499 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:00.001160 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:00.003435 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:00.004253 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:00.004353 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:00.016609 (+ 12256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:00.016627 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:00.016651 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:00.016737 (+ 86us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:00.020955 (+ 4218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:00.021596 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:00.021602 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:00.021604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:00.021901 (+ 297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:00.022305 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:00.022309 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:00.022311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:00.022708 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:00.022954 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:00.023291 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 23:27:00.023491 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18345,"thread_start_us":18858,"threads_started":1}
W20260808 23:27:00.024776 12150 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47020
W20260808 23:27:00.025532 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 11957: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:27:00.025897 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 11957
I20260808 23:27:00.059536 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 12162
Aug 08 23:27:00 dist-test-slave-6bbx krb5kdc[11786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231617, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.44.66@KRBTEST.COM
I20260808 23:27:00.073910 12787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:00.060295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59288 (local address 127.9.44.66:41849)
0808 23:27:00.060880 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:00.060885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:00.060900 (+ 15us) server_negotiation.cc:407] Connection header received
0808 23:27:00.060951 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:00.060954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:00.061015 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:00.061111 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:00.062287 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:00.063046 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:00.064381 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:00.064803 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:00.064913 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:00.068752 (+ 3839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:00.068765 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:00.068781 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:00.068818 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:00.071334 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:00.071975 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:00.071982 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:00.071984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:00.072115 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:00.072476 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:00.072480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:00.072482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:00.072823 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:00.073109 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:00.073374 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 23:27:00.073592 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":361,"threads_started":1}
W20260808 23:27:00.074672 12352 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59288
W20260808 23:27:00.075229 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 12162: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:27:00.075433 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 12162
I20260808 23:27:00.101536 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 12369
Aug 08 23:27:00 dist-test-slave-6bbx krb5kdc[11786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231617, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.44.67@KRBTEST.COM
I20260808 23:27:00.115489 12788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:00.102293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44950 (local address 127.9.44.67:33875)
0808 23:27:00.102787 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:00.102791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:00.102806 (+ 15us) server_negotiation.cc:407] Connection header received
0808 23:27:00.102858 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:00.102862 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:00.102918 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:00.103005 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:00.104544 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:00.105296 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:00.106418 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:00.106802 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:00.106885 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:00.110735 (+ 3850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:00.110750 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:00.110766 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:00.110804 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:00.113347 (+ 2543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:00.113900 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:00.113907 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:00.113909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:00.114032 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:00.114287 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:00.114290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:00.114292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:00.114601 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:00.114830 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:00.114996 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 23:27:00.115167 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":266,"threads_started":1}
W20260808 23:27:00.116146 12547 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44950
W20260808 23:27:00.116739 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 12369: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:27:00.116879 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 12369
I20260808 23:27:00.140944 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 11808
I20260808 23:27:00.142076 11921 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:00.255450 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 11808
2026-08-08T23:27:00Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8792 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:27:00 dist-test-slave-6bbx krb5kdc[12793](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:27:00 dist-test-slave-6bbx krb5kdc[12793](info): set up 2 sockets
Aug 08 23:27:00 dist-test-slave-6bbx krb5kdc[12793](info): commencing operation
krb5kdc: starting...
W20260808 23:27:01.462256 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.000s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:27:01 dist-test-slave-6bbx krb5kdc[12793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:27:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:27:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:27:01.638952 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:46839
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:46839
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:02.022652 12810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:02.022922 12810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:02.023018 12810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:02.030187 12810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:27:02.030288 12810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:02.030341 12810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:02.030398 12810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:27:02.030441 12810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:27:02.039918 12810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:46839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:46839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:02.041023 12810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:02.042256 12810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:02.053185 12817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:02.053256 12820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:02.053546 12818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:02.054473 12810 server_base.cc:1034] running on GCE node
I20260808 23:27:02.055166 12810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:02.056461 12810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:02.057659 12810 hybrid_clock.cc:648] HybridClock initialized: now 1786231622057610 us; error 36 us; skew 500 ppm
I20260808 23:27:02.057976 12810 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:02 dist-test-slave-6bbx krb5kdc[12793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231622, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:02.060976 12810 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:27:02.062346 12810 webserver.cc:460] Webserver started at http://127.9.44.126:43761/ using document root <none> and password file <none>
I20260808 23:27:02.062950 12810 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:02.063050 12810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:02.063315 12810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:02.066104 12810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "94dce6bbe15a483589b5b402b3b190ba"
format_stamp: "Formatted at 2026-08-08 23:27:02 on dist-test-slave-6bbx"
server_key: "fdac25b5f0083142827631aece8cdde2"
server_key_iv: "9d8c3675f68ece97075cab60f0def94f"
server_key_version: "encryptionkey@0"
I20260808 23:27:02.066820 12810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "94dce6bbe15a483589b5b402b3b190ba"
format_stamp: "Formatted at 2026-08-08 23:27:02 on dist-test-slave-6bbx"
server_key: "fdac25b5f0083142827631aece8cdde2"
server_key_iv: "9d8c3675f68ece97075cab60f0def94f"
server_key_version: "encryptionkey@0"
I20260808 23:27:02.071620 12810 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260808 23:27:02.074962 12830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:02.076246 12810 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:27:02.076385 12810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "94dce6bbe15a483589b5b402b3b190ba"
format_stamp: "Formatted at 2026-08-08 23:27:02 on dist-test-slave-6bbx"
server_key: "fdac25b5f0083142827631aece8cdde2"
server_key_iv: "9d8c3675f68ece97075cab60f0def94f"
server_key_version: "encryptionkey@0"
I20260808 23:27:02.076541 12810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:02.090041 12810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:02.093533 12810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:02.093752 12810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:02.110888 12810 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:46839
I20260808 23:27:02.110931 12897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:46839 every 8 connection(s)
I20260808 23:27:02.112146 12810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:27:02.113427 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 12810
I20260808 23:27:02.113649 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:27:02.114476 9393 external_mini_cluster.cc:1428] Setting key d7860f9fda221b68a85c1b84e4a6f7c8
I20260808 23:27:02.116575 12898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 23:27:02 dist-test-slave-6bbx krb5kdc[12793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:02.129053 12898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94dce6bbe15a483589b5b402b3b190ba: Bootstrap starting.
I20260808 23:27:02.132934 12898 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94dce6bbe15a483589b5b402b3b190ba: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:02.134088 12898 log.cc:824] T 00000000000000000000000000000000 P 94dce6bbe15a483589b5b402b3b190ba: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:02.137120 12898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94dce6bbe15a483589b5b402b3b190ba: No bootstrap required, opened a new log
I20260808 23:27:02.137357 12906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:02.116391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41568 (local address 127.9.44.126:46839)
0808 23:27:02.117140 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:02.117164 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:02.117211 (+ 47us) server_negotiation.cc:407] Connection header received
0808 23:27:02.119310 (+ 2099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:02.119338 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:02.119812 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:02.120229 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:02.121426 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:02.123443 (+ 2017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:02.124681 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:02.125649 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:02.125837 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:02.129397 (+ 3560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:02.129408 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:02.129424 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:02.129472 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:02.133091 (+ 3619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:02.133710 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:02.133718 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:02.133725 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:02.133938 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:02.134391 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:02.134396 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:02.134399 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:02.134971 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:02.135201 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:02.135750 (+ 549us) server_negotiation.cc:299] Negotiation successful
0808 23:27:02.136126 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":315,"threads_started":1}
I20260808 23:27:02.144201 12898 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94dce6bbe15a483589b5b402b3b190ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94dce6bbe15a483589b5b402b3b190ba" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 46839 } }
I20260808 23:27:02.144521 12898 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94dce6bbe15a483589b5b402b3b190ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:02.144652 12898 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94dce6bbe15a483589b5b402b3b190ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94dce6bbe15a483589b5b402b3b190ba, State: Initialized, Role: FOLLOWER
I20260808 23:27:02.145159 12898 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94dce6bbe15a483589b5b402b3b190ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94dce6bbe15a483589b5b402b3b190ba" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 46839 } }
I20260808 23:27:02.145359 12898 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94dce6bbe15a483589b5b402b3b190ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:27:02.145452 12898 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94dce6bbe15a483589b5b402b3b190ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:27:02.145578 12898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94dce6bbe15a483589b5b402b3b190ba [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:02.147464 12898 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94dce6bbe15a483589b5b402b3b190ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94dce6bbe15a483589b5b402b3b190ba" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 46839 } }
I20260808 23:27:02.147936 12898 leader_election.cc:302] T 00000000000000000000000000000000 P 94dce6bbe15a483589b5b402b3b190ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94dce6bbe15a483589b5b402b3b190ba; no voters:
I20260808 23:27:02.148449 12898 leader_election.cc:288] T 00000000000000000000000000000000 P 94dce6bbe15a483589b5b402b3b190ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:27:02.148588 12911 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94dce6bbe15a483589b5b402b3b190ba [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:27:02.149466 12911 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94dce6bbe15a483589b5b402b3b190ba [term 1 LEADER]: Becoming Leader. State: Replica: 94dce6bbe15a483589b5b402b3b190ba, State: Running, Role: LEADER
I20260808 23:27:02.149952 12911 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94dce6bbe15a483589b5b402b3b190ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94dce6bbe15a483589b5b402b3b190ba" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 46839 } }
I20260808 23:27:02.150327 12898 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94dce6bbe15a483589b5b402b3b190ba [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:27:02.152544 12913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94dce6bbe15a483589b5b402b3b190ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94dce6bbe15a483589b5b402b3b190ba. Latest consensus state: current_term: 1 leader_uuid: "94dce6bbe15a483589b5b402b3b190ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94dce6bbe15a483589b5b402b3b190ba" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 46839 } } }
I20260808 23:27:02.152565 12912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94dce6bbe15a483589b5b402b3b190ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94dce6bbe15a483589b5b402b3b190ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94dce6bbe15a483589b5b402b3b190ba" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 46839 } } }
I20260808 23:27:02.152976 12913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94dce6bbe15a483589b5b402b3b190ba [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:02.152976 12912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94dce6bbe15a483589b5b402b3b190ba [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:02.156816 12920 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:27:02.160671 12920 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:27:02.167843 12920 catalog_manager.cc:1347] Generated new cluster ID: 5e2bca95471a469082cc6fe5b9a35363
I20260808 23:27:02.167946 12920 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:27:02.183301 12920 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:27:02.185014 12920 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:27:02.191545 12920 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94dce6bbe15a483589b5b402b3b190ba: Generated new TSK 0
I20260808 23:27:02.192422 12920 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.44.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
I20260808 23:27:02.238795 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:0
--local_ip_for_outbound_sockets=127.9.44.65
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:46839
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:02.552802 12942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:02.553059 12942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:02.553193 12942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:02.560590 12942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:02.560698 12942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:02.560860 12942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:27:02.569965 12942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:46839
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:02.571069 12942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:02.572278 12942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:02.584636 12951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:02.584595 12954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:02.584679 12952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:02.585459 12942 server_base.cc:1034] running on GCE node
I20260808 23:27:02.586016 12942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:02.586838 12942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:02.588027 12942 hybrid_clock.cc:648] HybridClock initialized: now 1786231622587982 us; error 29 us; skew 500 ppm
I20260808 23:27:02.588304 12942 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:02 dist-test-slave-6bbx krb5kdc[12793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231622, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:02.591217 12942 init.cc:377] Logged in from keytab as kudu/127.9.44.65@KRBTEST.COM (short username kudu)
I20260808 23:27:02.592514 12942 webserver.cc:460] Webserver started at http://127.9.44.65:45667/ using document root <none> and password file <none>
I20260808 23:27:02.593120 12942 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:02.593240 12942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:02.593542 12942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:02.596660 12942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/data/instance:
uuid: "c431508b15a14c6ba88b37ac4a6c3b89"
format_stamp: "Formatted at 2026-08-08 23:27:02 on dist-test-slave-6bbx"
server_key: "0e087377c8d8a3e80724be4b4f2cdd82"
server_key_iv: "73649cefdfd387acb8486a2517c9419d"
server_key_version: "encryptionkey@0"
I20260808 23:27:02.597410 12942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance:
uuid: "c431508b15a14c6ba88b37ac4a6c3b89"
format_stamp: "Formatted at 2026-08-08 23:27:02 on dist-test-slave-6bbx"
server_key: "0e087377c8d8a3e80724be4b4f2cdd82"
server_key_iv: "73649cefdfd387acb8486a2517c9419d"
server_key_version: "encryptionkey@0"
I20260808 23:27:02.602155 12942 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 23:27:02.605412 12963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:02.606586 12942 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 23:27:02.606736 12942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "c431508b15a14c6ba88b37ac4a6c3b89"
format_stamp: "Formatted at 2026-08-08 23:27:02 on dist-test-slave-6bbx"
server_key: "0e087377c8d8a3e80724be4b4f2cdd82"
server_key_iv: "73649cefdfd387acb8486a2517c9419d"
server_key_version: "encryptionkey@0"
I20260808 23:27:02.606920 12942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:02.645272 12942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:02.648964 12942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:02.649255 12942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:02.650123 12942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:02.651450 12942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:02.651531 12942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:02.651618 12942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:02.651682 12942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:02.681483 12942 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:33711
I20260808 23:27:02.681541 13129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:33711 every 8 connection(s)
I20260808 23:27:02.682664 12942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:27:02.686028 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 12942
I20260808 23:27:02.686260 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance
I20260808 23:27:02.687075 9393 external_mini_cluster.cc:1428] Setting key 2422595de2f289c22d0e94616506f7a8
Aug 08 23:27:02 dist-test-slave-6bbx krb5kdc[12793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231622, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:02.701576 13132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:02.684958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:58139 (local address 127.9.44.126:46839)
0808 23:27:02.685407 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:02.685414 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:02.686472 (+ 1058us) server_negotiation.cc:407] Connection header received
0808 23:27:02.688608 (+ 2136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:02.688613 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:02.688689 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:02.688785 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:02.690155 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:02.690947 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:02.692465 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:02.692859 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:02.692966 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:02.696405 (+ 3439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:02.696410 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:02.696413 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:02.696443 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:02.698735 (+ 2292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:02.699394 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:02.699398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:02.699400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:02.699516 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:02.699964 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:02.699967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:02.699969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:02.700239 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:02.700383 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:02.701014 (+ 631us) server_negotiation.cc:299] Negotiation successful
0808 23:27:02.701242 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":211,"threads_started":1}
I20260808 23:27:02.702421 13134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:02.685507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:46839 (local address 127.9.44.65:58139)
0808 23:27:02.686301 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:02.686354 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:02.688357 (+ 2003us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:02.689016 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:02.689024 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:02.689480 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:02.689993 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:02.690010 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:02.691083 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:02.691087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:02.692330 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:02.692343 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:02.693066 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:02.693074 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:02.693276 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:02.693871 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:02.693900 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:02.696201 (+ 2301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:02.698861 (+ 2660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:02.698869 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:02.698883 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:02.699274 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:02.699614 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:02.699619 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:02.699623 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:02.699852 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:02.700371 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:02.700378 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:02.700669 (+ 291us) client_negotiation.cc:769] Sending connection context
0808 23:27:02.700894 (+ 225us) client_negotiation.cc:240] Negotiation successful
0808 23:27:02.701341 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":301,"threads_started":1}
I20260808 23:27:02.703888 13130 heartbeater.cc:347] Connected to a master server at 127.9.44.126:46839
I20260808 23:27:02.704162 13130 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:02.704819 13130 heartbeater.cc:511] Master 127.9.44.126:46839 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.44.66@KRBTEST.COM; defaulting to no policy
I20260808 23:27:02.706823 12850 ts_manager.cc:194] Registered new tserver with Master: c431508b15a14c6ba88b37ac4a6c3b89 (127.9.44.65:33711)
I20260808 23:27:02.708400 12850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.65@KRBTEST.COM'} at 127.9.44.65:58139
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
I20260808 23:27:02.729967 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:0
--local_ip_for_outbound_sockets=127.9.44.66
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:46839
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:03.043892 13138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:03.044162 13138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:03.044301 13138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:03.051162 13138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:03.051263 13138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:03.051411 13138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:27:03.060607 13138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:46839
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:03.061734 13138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:03.062968 13138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:03.075169 13144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:03.075129 13145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:03.075203 13148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:03.076174 13138 server_base.cc:1034] running on GCE node
I20260808 23:27:03.076879 13138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:03.077999 13138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:03.079274 13138 hybrid_clock.cc:648] HybridClock initialized: now 1786231623079209 us; error 49 us; skew 500 ppm
I20260808 23:27:03.079623 13138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:03 dist-test-slave-6bbx krb5kdc[12793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231623, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:03.083331 13138 init.cc:377] Logged in from keytab as kudu/127.9.44.66@KRBTEST.COM (short username kudu)
I20260808 23:27:03.085095 13138 webserver.cc:460] Webserver started at http://127.9.44.66:43079/ using document root <none> and password file <none>
I20260808 23:27:03.085741 13138 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:03.085862 13138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:03.086154 13138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:03.088996 13138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/data/instance:
uuid: "42a699f640a4406d9cd2ee265ef1b660"
format_stamp: "Formatted at 2026-08-08 23:27:03 on dist-test-slave-6bbx"
server_key: "0a9463794c351801509aace985ca1134"
server_key_iv: "6fb927399a85268987c93d4b3723e096"
server_key_version: "encryptionkey@0"
I20260808 23:27:03.089733 13138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance:
uuid: "42a699f640a4406d9cd2ee265ef1b660"
format_stamp: "Formatted at 2026-08-08 23:27:03 on dist-test-slave-6bbx"
server_key: "0a9463794c351801509aace985ca1134"
server_key_iv: "6fb927399a85268987c93d4b3723e096"
server_key_version: "encryptionkey@0"
I20260808 23:27:03.094482 13138 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 23:27:03.097692 13157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:03.098884 13138 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 23:27:03.099032 13138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "42a699f640a4406d9cd2ee265ef1b660"
format_stamp: "Formatted at 2026-08-08 23:27:03 on dist-test-slave-6bbx"
server_key: "0a9463794c351801509aace985ca1134"
server_key_iv: "6fb927399a85268987c93d4b3723e096"
server_key_version: "encryptionkey@0"
I20260808 23:27:03.099198 13138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:03.125846 13138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:03.129318 13138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:03.129560 13138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:03.130396 13138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:03.131588 13138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:03.131683 13138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:03.131803 13138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:03.131860 13138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:03.161110 13138 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:43289
I20260808 23:27:03.161166 13327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:43289 every 8 connection(s)
I20260808 23:27:03.162273 13138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:27:03.165189 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 13138
I20260808 23:27:03.165364 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance
I20260808 23:27:03.165784 9393 external_mini_cluster.cc:1428] Setting key 20be4953661f322b7ab086c3afe03b1e
Aug 08 23:27:03 dist-test-slave-6bbx krb5kdc[12793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231623, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:03.181180 13132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:03.164642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:43803 (local address 127.9.44.126:46839)
0808 23:27:03.164858 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:03.164865 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:03.165993 (+ 1128us) server_negotiation.cc:407] Connection header received
0808 23:27:03.168099 (+ 2106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:03.168105 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:03.168187 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:03.168299 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:03.169627 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:03.170649 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:03.172220 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:03.172757 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:03.172882 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:03.175945 (+ 3063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:03.175950 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:03.175953 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:03.175985 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:03.178345 (+ 2360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:03.179010 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:03.179014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:03.179016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:03.179143 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:03.179541 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:03.179544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:03.179547 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:03.179872 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:03.180019 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:03.180630 (+ 611us) server_negotiation.cc:299] Negotiation successful
0808 23:27:03.180850 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 23:27:03.182024 13331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:03.165047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:46839 (local address 127.9.44.66:43803)
0808 23:27:03.165830 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:03.165874 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:03.167854 (+ 1980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:03.168459 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:03.168467 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:03.168944 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:03.169457 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:03.169474 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:03.170835 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:03.170839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:03.172078 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:03.172095 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:03.172937 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:03.172942 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:03.173121 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:03.173714 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:03.173737 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:03.175744 (+ 2007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:03.178470 (+ 2726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:03.178479 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:03.178493 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:03.178892 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:03.179246 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:03.179251 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:03.179255 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:03.179429 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:03.179990 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:03.179997 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:03.180284 (+ 287us) client_negotiation.cc:769] Sending connection context
0808 23:27:03.180499 (+ 215us) client_negotiation.cc:240] Negotiation successful
0808 23:27:03.180997 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":333,"threads_started":1}
I20260808 23:27:03.183353 13329 heartbeater.cc:347] Connected to a master server at 127.9.44.126:46839
I20260808 23:27:03.183612 13329 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:03.184286 13329 heartbeater.cc:511] Master 127.9.44.126:46839 requested a full tablet report, sending...
I20260808 23:27:03.185669 12850 ts_manager.cc:194] Registered new tserver with Master: 42a699f640a4406d9cd2ee265ef1b660 (127.9.44.66:43289)
WARNING: no policy specified for kudu/127.9.44.67@KRBTEST.COM; defaulting to no policy
I20260808 23:27:03.186724 12850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.66@KRBTEST.COM'} at 127.9.44.66:43803
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
I20260808 23:27:03.208658 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:0
--local_ip_for_outbound_sockets=127.9.44.67
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:46839
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:03.522861 13336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:03.523111 13336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:03.523207 13336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:03.530122 13336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:03.530215 13336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:03.530324 13336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:27:03.539710 13336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:46839
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:03.540817 13336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:03.542023 13336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:03.554188 13345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:03.554219 13343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:03.554381 13347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:03.555001 13336 server_base.cc:1034] running on GCE node
I20260808 23:27:03.555639 13336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:03.556631 13336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:03.557828 13336 hybrid_clock.cc:648] HybridClock initialized: now 1786231623557773 us; error 48 us; skew 500 ppm
I20260808 23:27:03.558125 13336 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:03 dist-test-slave-6bbx krb5kdc[12793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231623, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:03.561088 13336 init.cc:377] Logged in from keytab as kudu/127.9.44.67@KRBTEST.COM (short username kudu)
I20260808 23:27:03.562383 13336 webserver.cc:460] Webserver started at http://127.9.44.67:41519/ using document root <none> and password file <none>
I20260808 23:27:03.563024 13336 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:03.563143 13336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:03.563431 13336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:03.566540 13336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/data/instance:
uuid: "50bce678999a4567b4c7bb7a9e283633"
format_stamp: "Formatted at 2026-08-08 23:27:03 on dist-test-slave-6bbx"
server_key: "d2a9903cf849c3c352b1c78cab80a8ee"
server_key_iv: "f8a572794d6293984d464d0a957b530e"
server_key_version: "encryptionkey@0"
I20260808 23:27:03.567266 13336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance:
uuid: "50bce678999a4567b4c7bb7a9e283633"
format_stamp: "Formatted at 2026-08-08 23:27:03 on dist-test-slave-6bbx"
server_key: "d2a9903cf849c3c352b1c78cab80a8ee"
server_key_iv: "f8a572794d6293984d464d0a957b530e"
server_key_version: "encryptionkey@0"
I20260808 23:27:03.572089 13336 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260808 23:27:03.575320 13355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:03.576527 13336 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 23:27:03.576670 13336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "50bce678999a4567b4c7bb7a9e283633"
format_stamp: "Formatted at 2026-08-08 23:27:03 on dist-test-slave-6bbx"
server_key: "d2a9903cf849c3c352b1c78cab80a8ee"
server_key_iv: "f8a572794d6293984d464d0a957b530e"
server_key_version: "encryptionkey@0"
I20260808 23:27:03.576843 13336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:03.590723 13336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:03.594739 13336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:03.595008 13336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:03.595963 13336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:03.597218 13336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:03.597297 13336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:03.597414 13336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:03.597472 13336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:03.627446 13336 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:41723
I20260808 23:27:03.627486 13522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:41723 every 8 connection(s)
I20260808 23:27:03.628659 13336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:27:03.633780 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 13336
I20260808 23:27:03.633957 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance
I20260808 23:27:03.634378 9393 external_mini_cluster.cc:1428] Setting key f883ba16d263e9e9789beda681aa82c4
Aug 08 23:27:03 dist-test-slave-6bbx krb5kdc[12793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231623, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:03.647413 13132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:03.631029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:34135 (local address 127.9.44.126:46839)
0808 23:27:03.631198 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:03.631202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:03.632328 (+ 1126us) server_negotiation.cc:407] Connection header received
0808 23:27:03.634101 (+ 1773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:03.634107 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:03.634191 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:03.634289 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:03.635990 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:03.636991 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:03.638586 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:03.638970 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:03.639059 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:03.642563 (+ 3504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:03.642567 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:03.642570 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:03.642602 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:03.645029 (+ 2427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:03.645631 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:03.645634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:03.645636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:03.645752 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:03.646085 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:03.646087 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:03.646089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:03.646349 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:03.646456 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:03.647001 (+ 545us) server_negotiation.cc:299] Negotiation successful
0808 23:27:03.647155 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 23:27:03.648689 13525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:03.631475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:46839 (local address 127.9.44.67:34135)
0808 23:27:03.632178 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:03.632218 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:03.633829 (+ 1611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:03.634537 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:03.634546 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:03.635173 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:03.635829 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:03.635849 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:03.637157 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:03.637163 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:03.638460 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:03.638474 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:03.639160 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:03.639167 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:03.639476 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:03.640333 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:03.640363 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:03.642371 (+ 2008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:03.645134 (+ 2763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:03.645140 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:03.645150 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:03.645541 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:03.645826 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:03.645828 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:03.645831 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:03.646008 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:03.646436 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:03.646441 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:03.646719 (+ 278us) client_negotiation.cc:769] Sending connection context
0808 23:27:03.646933 (+ 214us) client_negotiation.cc:240] Negotiation successful
0808 23:27:03.647276 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":291,"threads_started":1}
I20260808 23:27:03.649909 13523 heartbeater.cc:347] Connected to a master server at 127.9.44.126:46839
I20260808 23:27:03.650187 13523 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:03.650876 13523 heartbeater.cc:511] Master 127.9.44.126:46839 requested a full tablet report, sending...
I20260808 23:27:03.652211 12850 ts_manager.cc:194] Registered new tserver with Master: 50bce678999a4567b4c7bb7a9e283633 (127.9.44.67:41723)
I20260808 23:27:03.652962 12850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.67@KRBTEST.COM'} at 127.9.44.67:34135
I20260808 23:27:03.661353 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 23:27:03.694042 13132 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:03.691908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41584 (local address 127.9.44.126:46839)
0808 23:27:03.692066 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:03.692071 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:03.692568 (+ 497us) server_negotiation.cc:407] Connection header received
0808 23:27:03.692706 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:03.692710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:03.692771 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:03.692853 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:03.693843 (+ 990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41584: BlockingRecv error: recv got EOF from 127.0.0.1:41584 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 23:27:03.697117 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 12942
W20260808 23:27:03.699397 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 12942: Not authorized: Client connection negotiation failed: client connection to 127.9.44.65:33711: server requires authentication, but client does not have Kerberos credentials available
I20260808 23:27:03.699571 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 12942
I20260808 23:27:03.721153 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 13138
W20260808 23:27:03.723362 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 13138: Not authorized: Client connection negotiation failed: client connection to 127.9.44.66:43289: server requires authentication, but client does not have Kerberos credentials available
I20260808 23:27:03.723495 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 13138
I20260808 23:27:03.738266 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 13336
W20260808 23:27:03.740756 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 13336: Not authorized: Client connection negotiation failed: client connection to 127.9.44.67:41723: server requires authentication, but client does not have Kerberos credentials available
I20260808 23:27:03.740893 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 13336
I20260808 23:27:03.759636 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 12810
I20260808 23:27:03.760788 12892 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:03.860738 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 12810
2026-08-08T23:27:03Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3602 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:27:03 dist-test-slave-6bbx krb5kdc[13549](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:27:03 dist-test-slave-6bbx krb5kdc[13549](info): set up 2 sockets
Aug 08 23:27:03 dist-test-slave-6bbx krb5kdc[13549](info): commencing operation
krb5kdc: starting...
W20260808 23:27:05.039407 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:27:05 dist-test-slave-6bbx krb5kdc[13549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:27:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:27:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:27:05.226433 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:35659
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:35659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:05.532275 13565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:05.532555 13565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:05.532691 13565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:05.539646 13565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:27:05.539765 13565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:05.539844 13565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:05.539924 13565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:27:05.539991 13565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:27:05.549399 13565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:35659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:35659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:05.550570 13565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:05.551802 13565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:05.563006 13571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:05.562887 13572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:05.563009 13574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:05.563911 13565 server_base.cc:1034] running on GCE node
I20260808 23:27:05.564633 13565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:05.565855 13565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:05.567052 13565 hybrid_clock.cc:648] HybridClock initialized: now 1786231625566998 us; error 51 us; skew 500 ppm
I20260808 23:27:05.567364 13565 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:05 dist-test-slave-6bbx krb5kdc[13549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231625, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:05.573239 13565 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:27:05.574602 13565 webserver.cc:460] Webserver started at http://127.9.44.126:40851/ using document root <none> and password file <none>
I20260808 23:27:05.575213 13565 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:05.575331 13565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:05.575618 13565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:05.578485 13565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "5c59dfac4f5949a8a9223d885ec2ee2a"
format_stamp: "Formatted at 2026-08-08 23:27:05 on dist-test-slave-6bbx"
server_key: "83ee87a6a484ee32db0f3fe279e8d38b"
server_key_iv: "023d538e7658c5ce14b5ad56a1bc0158"
server_key_version: "encryptionkey@0"
I20260808 23:27:05.579245 13565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "5c59dfac4f5949a8a9223d885ec2ee2a"
format_stamp: "Formatted at 2026-08-08 23:27:05 on dist-test-slave-6bbx"
server_key: "83ee87a6a484ee32db0f3fe279e8d38b"
server_key_iv: "023d538e7658c5ce14b5ad56a1bc0158"
server_key_version: "encryptionkey@0"
I20260808 23:27:05.584038 13565 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260808 23:27:05.587281 13587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:05.588491 13565 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:27:05.588649 13565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "5c59dfac4f5949a8a9223d885ec2ee2a"
format_stamp: "Formatted at 2026-08-08 23:27:05 on dist-test-slave-6bbx"
server_key: "83ee87a6a484ee32db0f3fe279e8d38b"
server_key_iv: "023d538e7658c5ce14b5ad56a1bc0158"
server_key_version: "encryptionkey@0"
I20260808 23:27:05.588821 13565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:05.648752 13565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:05.652659 13565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:05.652905 13565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:05.669889 13565 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:35659
I20260808 23:27:05.669942 13663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:35659 every 8 connection(s)
I20260808 23:27:05.671123 13565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:27:05.674634 13664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:05.680886 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 13565
I20260808 23:27:05.681089 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:27:05.681883 9393 external_mini_cluster.cc:1428] Setting key a9c4ad8c8eaec418f12515c853c2f9a1
I20260808 23:27:05.685374 13664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c59dfac4f5949a8a9223d885ec2ee2a: Bootstrap starting.
I20260808 23:27:05.688872 13664 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c59dfac4f5949a8a9223d885ec2ee2a: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:05.689865 13664 log.cc:824] T 00000000000000000000000000000000 P 5c59dfac4f5949a8a9223d885ec2ee2a: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:05.692736 13664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c59dfac4f5949a8a9223d885ec2ee2a: No bootstrap required, opened a new log
Aug 08 23:27:05 dist-test-slave-6bbx krb5kdc[13549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:05.700644 13664 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c59dfac4f5949a8a9223d885ec2ee2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c59dfac4f5949a8a9223d885ec2ee2a" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35659 } }
I20260808 23:27:05.700949 13664 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c59dfac4f5949a8a9223d885ec2ee2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:05.701058 13664 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c59dfac4f5949a8a9223d885ec2ee2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c59dfac4f5949a8a9223d885ec2ee2a, State: Initialized, Role: FOLLOWER
I20260808 23:27:05.701678 13664 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c59dfac4f5949a8a9223d885ec2ee2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c59dfac4f5949a8a9223d885ec2ee2a" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35659 } }
I20260808 23:27:05.701975 13664 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c59dfac4f5949a8a9223d885ec2ee2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:27:05.702173 13664 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c59dfac4f5949a8a9223d885ec2ee2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:27:05.702375 13664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c59dfac4f5949a8a9223d885ec2ee2a [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:05.703492 13667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:05.683693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49898 (local address 127.9.44.126:35659)
0808 23:27:05.684423 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:05.684434 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:05.684472 (+ 38us) server_negotiation.cc:407] Connection header received
0808 23:27:05.686133 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:05.686157 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:05.686486 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:05.686815 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:05.688607 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:05.689894 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:05.691396 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:05.692111 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:05.692238 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:05.696597 (+ 4359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:05.696604 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:05.696617 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:05.696649 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:05.699506 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:05.700150 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:05.700157 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:05.700163 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:05.700321 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:05.700679 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:05.700682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:05.700684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:05.701252 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:05.701539 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:05.701934 (+ 395us) server_negotiation.cc:299] Negotiation successful
0808 23:27:05.702508 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":315,"threads_started":1}
I20260808 23:27:05.704603 13664 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c59dfac4f5949a8a9223d885ec2ee2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c59dfac4f5949a8a9223d885ec2ee2a" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35659 } }
I20260808 23:27:05.705072 13664 leader_election.cc:302] T 00000000000000000000000000000000 P 5c59dfac4f5949a8a9223d885ec2ee2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c59dfac4f5949a8a9223d885ec2ee2a; no voters:
I20260808 23:27:05.705651 13664 leader_election.cc:288] T 00000000000000000000000000000000 P 5c59dfac4f5949a8a9223d885ec2ee2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:27:05.705749 13669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c59dfac4f5949a8a9223d885ec2ee2a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:27:05.706866 13669 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c59dfac4f5949a8a9223d885ec2ee2a [term 1 LEADER]: Becoming Leader. State: Replica: 5c59dfac4f5949a8a9223d885ec2ee2a, State: Running, Role: LEADER
I20260808 23:27:05.707131 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:27:05.707458 13669 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c59dfac4f5949a8a9223d885ec2ee2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c59dfac4f5949a8a9223d885ec2ee2a" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35659 } }
I20260808 23:27:05.707626 13664 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c59dfac4f5949a8a9223d885ec2ee2a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:27:05.710376 13670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c59dfac4f5949a8a9223d885ec2ee2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c59dfac4f5949a8a9223d885ec2ee2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c59dfac4f5949a8a9223d885ec2ee2a" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35659 } } }
I20260808 23:27:05.710806 13670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c59dfac4f5949a8a9223d885ec2ee2a [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:05.710412 13671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c59dfac4f5949a8a9223d885ec2ee2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c59dfac4f5949a8a9223d885ec2ee2a. Latest consensus state: current_term: 1 leader_uuid: "5c59dfac4f5949a8a9223d885ec2ee2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c59dfac4f5949a8a9223d885ec2ee2a" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35659 } } }
I20260808 23:27:05.711383 13671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c59dfac4f5949a8a9223d885ec2ee2a [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:05.713635 13677 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:27:05.718411 13677 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:27:05.728018 13677 catalog_manager.cc:1347] Generated new cluster ID: 9b64ba8e55ab4b939a0963f31654bfbe
I20260808 23:27:05.728116 13677 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:27:05.740216 13677 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:27:05.741937 13677 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:27:05.748395 13677 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c59dfac4f5949a8a9223d885ec2ee2a: Generated new TSK 0
I20260808 23:27:05.749392 13677 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:27:05.754725 13667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:05.747251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49910 (local address 127.9.44.126:35659)
0808 23:27:05.747420 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:05.747427 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:05.748193 (+ 766us) server_negotiation.cc:407] Connection header received
0808 23:27:05.748417 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:05.748423 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:05.748504 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:05.748589 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:05.749705 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:05.750442 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:05.751709 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:05.752151 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:05.752274 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:05.753260 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:05.753265 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:05.753268 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:27:05.754003 (+ 735us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:05.754067 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:27:05.754098 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:05.754189 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:05.754333 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 23:27:05.754433 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 23:27:05.761696 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 13565
I20260808 23:27:05.762727 13658 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:05.849450 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 13565
2026-08-08T23:27:05Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1985 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:27:05 dist-test-slave-6bbx krb5kdc[13709](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:27:05 dist-test-slave-6bbx krb5kdc[13709](info): set up 2 sockets
Aug 08 23:27:05 dist-test-slave-6bbx krb5kdc[13709](info): commencing operation
krb5kdc: starting...
W20260808 23:27:07.016176 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.003s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:27:07 dist-test-slave-6bbx krb5kdc[13709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:27:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:27:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:27:07.197996 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:40453
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:40453
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:07.505355 13728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:07.505633 13728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:07.505767 13728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:07.512804 13728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:27:07.512913 13728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:07.512991 13728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:07.513070 13728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:27:07.513139 13728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:27:07.522619 13728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:40453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:40453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:07.523838 13728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:07.525046 13728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:07.536512 13739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:07.536470 13744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:07.536558 13741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:07.537319 13728 server_base.cc:1034] running on GCE node
I20260808 23:27:07.538031 13728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:07.539283 13728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:07.540467 13728 hybrid_clock.cc:648] HybridClock initialized: now 1786231627540420 us; error 37 us; skew 500 ppm
I20260808 23:27:07.540794 13728 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:07 dist-test-slave-6bbx krb5kdc[13709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231627, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:07.543725 13728 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:27:07.545074 13728 webserver.cc:460] Webserver started at http://127.9.44.126:36517/ using document root <none> and password file <none>
I20260808 23:27:07.545670 13728 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:07.545786 13728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:07.546067 13728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:07.548898 13728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "cafe9ac77bc64530830c7e551f8c91c8"
format_stamp: "Formatted at 2026-08-08 23:27:07 on dist-test-slave-6bbx"
server_key: "386c40dd3d7e10e2dc5d77ca6bd60b13"
server_key_iv: "bab1d40eb19f48ea1cbe2502375978b5"
server_key_version: "encryptionkey@0"
I20260808 23:27:07.549597 13728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "cafe9ac77bc64530830c7e551f8c91c8"
format_stamp: "Formatted at 2026-08-08 23:27:07 on dist-test-slave-6bbx"
server_key: "386c40dd3d7e10e2dc5d77ca6bd60b13"
server_key_iv: "bab1d40eb19f48ea1cbe2502375978b5"
server_key_version: "encryptionkey@0"
I20260808 23:27:07.554363 13728 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 23:27:07.557545 13754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:07.558668 13728 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 23:27:07.558827 13728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "cafe9ac77bc64530830c7e551f8c91c8"
format_stamp: "Formatted at 2026-08-08 23:27:07 on dist-test-slave-6bbx"
server_key: "386c40dd3d7e10e2dc5d77ca6bd60b13"
server_key_iv: "bab1d40eb19f48ea1cbe2502375978b5"
server_key_version: "encryptionkey@0"
I20260808 23:27:07.558993 13728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:07.577726 13728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:07.581559 13728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:07.581813 13728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:07.598591 13728 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:40453
I20260808 23:27:07.598632 13831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:40453 every 8 connection(s)
I20260808 23:27:07.599838 13728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:27:07.602756 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 13728
I20260808 23:27:07.602981 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:27:07.603528 13832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:07.603839 9393 external_mini_cluster.cc:1428] Setting key 12466af717543ac8f6775de041fc2139
I20260808 23:27:07.615366 13832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cafe9ac77bc64530830c7e551f8c91c8: Bootstrap starting.
Aug 08 23:27:07 dist-test-slave-6bbx krb5kdc[13709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:07.618453 13832 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cafe9ac77bc64530830c7e551f8c91c8: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:07.619479 13832 log.cc:824] T 00000000000000000000000000000000 P cafe9ac77bc64530830c7e551f8c91c8: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:07.622112 13832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cafe9ac77bc64530830c7e551f8c91c8: No bootstrap required, opened a new log
I20260808 23:27:07.624240 13835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:07.605809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46188 (local address 127.9.44.126:40453)
0808 23:27:07.606619 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:07.606630 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:07.606667 (+ 37us) server_negotiation.cc:407] Connection header received
0808 23:27:07.608232 (+ 1565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:07.608254 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:07.608608 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:07.608968 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:07.610350 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:07.611544 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:07.612752 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:07.613440 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:07.613623 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:07.617442 (+ 3819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:07.617449 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:07.617461 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:07.617493 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:07.620381 (+ 2888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:07.621045 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:07.621052 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:07.621057 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:07.621217 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:07.621566 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:07.621570 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:07.621572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:07.622047 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:07.622379 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:07.622781 (+ 402us) server_negotiation.cc:299] Negotiation successful
0808 23:27:07.623152 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":346,"threads_started":1}
I20260808 23:27:07.630501 13832 raft_consensus.cc:348] T 00000000000000000000000000000000 P cafe9ac77bc64530830c7e551f8c91c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cafe9ac77bc64530830c7e551f8c91c8" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 40453 } }
I20260808 23:27:07.630820 13832 raft_consensus.cc:374] T 00000000000000000000000000000000 P cafe9ac77bc64530830c7e551f8c91c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:07.630898 13832 raft_consensus.cc:729] T 00000000000000000000000000000000 P cafe9ac77bc64530830c7e551f8c91c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cafe9ac77bc64530830c7e551f8c91c8, State: Initialized, Role: FOLLOWER
I20260808 23:27:07.631392 13832 consensus_queue.cc:260] T 00000000000000000000000000000000 P cafe9ac77bc64530830c7e551f8c91c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cafe9ac77bc64530830c7e551f8c91c8" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 40453 } }
I20260808 23:27:07.631594 13832 raft_consensus.cc:388] T 00000000000000000000000000000000 P cafe9ac77bc64530830c7e551f8c91c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:27:07.631729 13832 raft_consensus.cc:482] T 00000000000000000000000000000000 P cafe9ac77bc64530830c7e551f8c91c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:27:07.631892 13832 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cafe9ac77bc64530830c7e551f8c91c8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:07.633968 13832 raft_consensus.cc:504] T 00000000000000000000000000000000 P cafe9ac77bc64530830c7e551f8c91c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cafe9ac77bc64530830c7e551f8c91c8" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 40453 } }
I20260808 23:27:07.634392 13832 leader_election.cc:302] T 00000000000000000000000000000000 P cafe9ac77bc64530830c7e551f8c91c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cafe9ac77bc64530830c7e551f8c91c8; no voters:
I20260808 23:27:07.634963 13832 leader_election.cc:288] T 00000000000000000000000000000000 P cafe9ac77bc64530830c7e551f8c91c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:27:07.635080 13838 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cafe9ac77bc64530830c7e551f8c91c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:27:07.635769 13838 raft_consensus.cc:686] T 00000000000000000000000000000000 P cafe9ac77bc64530830c7e551f8c91c8 [term 1 LEADER]: Becoming Leader. State: Replica: cafe9ac77bc64530830c7e551f8c91c8, State: Running, Role: LEADER
I20260808 23:27:07.636309 13838 consensus_queue.cc:237] T 00000000000000000000000000000000 P cafe9ac77bc64530830c7e551f8c91c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cafe9ac77bc64530830c7e551f8c91c8" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 40453 } }
I20260808 23:27:07.636895 13832 sys_catalog.cc:564] T 00000000000000000000000000000000 P cafe9ac77bc64530830c7e551f8c91c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:27:07.638933 13840 sys_catalog.cc:455] T 00000000000000000000000000000000 P cafe9ac77bc64530830c7e551f8c91c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cafe9ac77bc64530830c7e551f8c91c8. Latest consensus state: current_term: 1 leader_uuid: "cafe9ac77bc64530830c7e551f8c91c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cafe9ac77bc64530830c7e551f8c91c8" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 40453 } } }
I20260808 23:27:07.639274 13840 sys_catalog.cc:458] T 00000000000000000000000000000000 P cafe9ac77bc64530830c7e551f8c91c8 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:07.639755 13839 sys_catalog.cc:455] T 00000000000000000000000000000000 P cafe9ac77bc64530830c7e551f8c91c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cafe9ac77bc64530830c7e551f8c91c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cafe9ac77bc64530830c7e551f8c91c8" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 40453 } } }
I20260808 23:27:07.640163 13839 sys_catalog.cc:458] T 00000000000000000000000000000000 P cafe9ac77bc64530830c7e551f8c91c8 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:07.641049 13843 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:27:07.646063 13843 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:27:07.654516 13843 catalog_manager.cc:1347] Generated new cluster ID: 2f42feaa0b434dd29b4f90c936377f72
I20260808 23:27:07.654603 13843 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:27:07.667338 13843 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:27:07.668526 13843 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:27:07.674316 13843 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cafe9ac77bc64530830c7e551f8c91c8: Generated new TSK 0
I20260808 23:27:07.674957 13843 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.44.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
I20260808 23:27:07.728998 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:0
--local_ip_for_outbound_sockets=127.9.44.65
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:40453
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:08.057744 13870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:08.057991 13870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:08.058081 13870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:08.065146 13870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:08.065243 13870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:08.065357 13870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:27:08.074513 13870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:40453
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:08.075640 13870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:08.076879 13870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:08.089098 13877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:08.089133 13876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:08.089160 13879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:08.089887 13870 server_base.cc:1034] running on GCE node
I20260808 23:27:08.090406 13870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:08.091244 13870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:08.092427 13870 hybrid_clock.cc:648] HybridClock initialized: now 1786231628092371 us; error 44 us; skew 500 ppm
I20260808 23:27:08.092696 13870 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:08 dist-test-slave-6bbx krb5kdc[13709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231628, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:08.095607 13870 init.cc:377] Logged in from keytab as kudu/127.9.44.65@KRBTEST.COM (short username kudu)
I20260808 23:27:08.096908 13870 webserver.cc:460] Webserver started at http://127.9.44.65:40889/ using document root <none> and password file <none>
I20260808 23:27:08.097546 13870 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:08.097653 13870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:08.097942 13870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:08.100714 13870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/data/instance:
uuid: "0d4316b191ee4b2ab306ca5521a973f3"
format_stamp: "Formatted at 2026-08-08 23:27:08 on dist-test-slave-6bbx"
server_key: "387a5e40a5ff22eca233ac011621281f"
server_key_iv: "aa61a8605a3276493e23a14b961d2940"
server_key_version: "encryptionkey@0"
I20260808 23:27:08.101446 13870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance:
uuid: "0d4316b191ee4b2ab306ca5521a973f3"
format_stamp: "Formatted at 2026-08-08 23:27:08 on dist-test-slave-6bbx"
server_key: "387a5e40a5ff22eca233ac011621281f"
server_key_iv: "aa61a8605a3276493e23a14b961d2940"
server_key_version: "encryptionkey@0"
I20260808 23:27:08.106163 13870 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 23:27:08.109411 13889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:08.110579 13870 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 23:27:08.110719 13870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "0d4316b191ee4b2ab306ca5521a973f3"
format_stamp: "Formatted at 2026-08-08 23:27:08 on dist-test-slave-6bbx"
server_key: "387a5e40a5ff22eca233ac011621281f"
server_key_iv: "aa61a8605a3276493e23a14b961d2940"
server_key_version: "encryptionkey@0"
I20260808 23:27:08.110903 13870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:08.131923 13870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:08.135419 13870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:08.135696 13870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:08.136575 13870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:08.137907 13870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:08.137989 13870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:08.138108 13870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:08.138168 13870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:08.167616 13870 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:35357
I20260808 23:27:08.167698 14058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:35357 every 8 connection(s)
I20260808 23:27:08.168809 13870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:27:08.170693 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 13870
I20260808 23:27:08.170886 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance
I20260808 23:27:08.171689 9393 external_mini_cluster.cc:1428] Setting key 1250746a8fd508c68819862b3c0b0235
Aug 08 23:27:08 dist-test-slave-6bbx krb5kdc[13709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231628, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:08.188712 14062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:08.171284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:38281 (local address 127.9.44.126:40453)
0808 23:27:08.171804 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:08.171812 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:08.172837 (+ 1025us) server_negotiation.cc:407] Connection header received
0808 23:27:08.174791 (+ 1954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:08.174797 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:08.174876 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:08.174990 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:08.176407 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:08.177341 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:08.178812 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:08.179321 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:08.179436 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:08.182726 (+ 3290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:08.182733 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:08.182737 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:08.182774 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:08.185841 (+ 3067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:08.186499 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:08.186504 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:08.186507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:08.186660 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:08.187036 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:08.187040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:08.187043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:08.187367 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:08.187494 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:08.188200 (+ 706us) server_negotiation.cc:299] Negotiation successful
0808 23:27:08.188397 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":243,"threads_started":1}
I20260808 23:27:08.189366 14064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:08.171874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:40453 (local address 127.9.44.65:38281)
0808 23:27:08.172677 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:08.172723 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:08.174485 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:08.175165 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:08.175173 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:08.175700 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:08.176242 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:08.176261 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:08.177463 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:08.177467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:08.178663 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:08.178675 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:08.179493 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:08.179498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:08.179729 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:08.180330 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:08.180352 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:08.182523 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:08.185964 (+ 3441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:08.185970 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:08.185981 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:08.186366 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:08.186759 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:08.186762 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:08.186764 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:08.186929 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:08.187470 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:08.187475 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:08.187804 (+ 329us) client_negotiation.cc:769] Sending connection context
0808 23:27:08.188014 (+ 210us) client_negotiation.cc:240] Negotiation successful
0808 23:27:08.188344 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":326,"threads_started":1}
I20260808 23:27:08.190804 14059 heartbeater.cc:347] Connected to a master server at 127.9.44.126:40453
I20260808 23:27:08.191113 14059 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:08.192011 14059 heartbeater.cc:511] Master 127.9.44.126:40453 requested a full tablet report, sending...
I20260808 23:27:08.194188 13777 ts_manager.cc:194] Registered new tserver with Master: 0d4316b191ee4b2ab306ca5521a973f3 (127.9.44.65:35357)
WARNING: no policy specified for kudu/127.9.44.66@KRBTEST.COM; defaulting to no policy
I20260808 23:27:08.195591 13777 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.65@KRBTEST.COM'} at 127.9.44.65:38281
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
I20260808 23:27:08.217967 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:0
--local_ip_for_outbound_sockets=127.9.44.66
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:40453
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:08.527215 14068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:08.527493 14068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:08.527619 14068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:08.534512 14068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:08.534626 14068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:08.534772 14068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:27:08.543931 14068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:40453
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:08.545039 14068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:08.546214 14068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:08.558382 14073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:08.558471 14079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:08.558380 14075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:08.559060 14068 server_base.cc:1034] running on GCE node
I20260808 23:27:08.559610 14068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:08.560557 14068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:08.561751 14068 hybrid_clock.cc:648] HybridClock initialized: now 1786231628561688 us; error 40 us; skew 500 ppm
I20260808 23:27:08.562060 14068 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:08 dist-test-slave-6bbx krb5kdc[13709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231628, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:08.565104 14068 init.cc:377] Logged in from keytab as kudu/127.9.44.66@KRBTEST.COM (short username kudu)
I20260808 23:27:08.566375 14068 webserver.cc:460] Webserver started at http://127.9.44.66:42203/ using document root <none> and password file <none>
I20260808 23:27:08.567008 14068 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:08.567119 14068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:08.567396 14068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:08.570130 14068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/data/instance:
uuid: "d6bf3145f2a54ec49f27d5a65912489c"
format_stamp: "Formatted at 2026-08-08 23:27:08 on dist-test-slave-6bbx"
server_key: "c5fd05401f9184a284829be1f515127e"
server_key_iv: "8364556f1424041c67eb72591b20c21a"
server_key_version: "encryptionkey@0"
I20260808 23:27:08.570874 14068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance:
uuid: "d6bf3145f2a54ec49f27d5a65912489c"
format_stamp: "Formatted at 2026-08-08 23:27:08 on dist-test-slave-6bbx"
server_key: "c5fd05401f9184a284829be1f515127e"
server_key_iv: "8364556f1424041c67eb72591b20c21a"
server_key_version: "encryptionkey@0"
I20260808 23:27:08.575574 14068 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 23:27:08.578692 14089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:08.579945 14068 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 23:27:08.580094 14068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "d6bf3145f2a54ec49f27d5a65912489c"
format_stamp: "Formatted at 2026-08-08 23:27:08 on dist-test-slave-6bbx"
server_key: "c5fd05401f9184a284829be1f515127e"
server_key_iv: "8364556f1424041c67eb72591b20c21a"
server_key_version: "encryptionkey@0"
I20260808 23:27:08.580258 14068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:08.594714 14068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:08.598770 14068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:08.599057 14068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:08.600101 14068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:08.601601 14068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:08.601709 14068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:08.601837 14068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:08.601897 14068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:08.633920 14068 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:39623
I20260808 23:27:08.633962 14253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:39623 every 8 connection(s)
I20260808 23:27:08.635051 14068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:27:08.642405 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 14068
I20260808 23:27:08.642557 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance
I20260808 23:27:08.643065 9393 external_mini_cluster.cc:1428] Setting key efd72f6a35bbae88aea8b1cbdf3f3854
Aug 08 23:27:08 dist-test-slave-6bbx krb5kdc[13709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231628, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:08.654357 14062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:08.637361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:51789 (local address 127.9.44.126:40453)
0808 23:27:08.637568 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:08.637575 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:08.638676 (+ 1101us) server_negotiation.cc:407] Connection header received
0808 23:27:08.640394 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:08.640398 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:08.640467 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:08.640541 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:08.641952 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:08.642806 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:08.644701 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:08.645105 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:08.645196 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:08.649069 (+ 3873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:08.649074 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:08.649077 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:08.649109 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:08.651485 (+ 2376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:08.652267 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:08.652271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:08.652273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:08.652397 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:08.652811 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:08.652814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:08.652815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:08.653068 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:08.653220 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:08.653792 (+ 572us) server_negotiation.cc:299] Negotiation successful
0808 23:27:08.654008 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 23:27:08.655000 14258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:08.637828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:40453 (local address 127.9.44.66:51789)
0808 23:27:08.638534 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:08.638570 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:08.640142 (+ 1572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:08.640724 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:08.640731 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:08.641197 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:08.641794 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:08.641818 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:08.642971 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:08.642977 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:08.644565 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:08.644578 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:08.645324 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:08.645331 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:08.645595 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:08.646224 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:08.646253 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:08.648845 (+ 2592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:08.651624 (+ 2779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:08.651632 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:08.651645 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:08.652158 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:08.652488 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:08.652492 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:08.652495 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:08.652725 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:08.653190 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:08.653196 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:08.653463 (+ 267us) client_negotiation.cc:769] Sending connection context
0808 23:27:08.653663 (+ 200us) client_negotiation.cc:240] Negotiation successful
0808 23:27:08.654042 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":298,"threads_started":1}
I20260808 23:27:08.656296 14255 heartbeater.cc:347] Connected to a master server at 127.9.44.126:40453
I20260808 23:27:08.656553 14255 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:08.657231 14255 heartbeater.cc:511] Master 127.9.44.126:40453 requested a full tablet report, sending...
I20260808 23:27:08.658568 13777 ts_manager.cc:194] Registered new tserver with Master: d6bf3145f2a54ec49f27d5a65912489c (127.9.44.66:39623)
I20260808 23:27:08.659338 13777 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.66@KRBTEST.COM'} at 127.9.44.66:51789
WARNING: no policy specified for kudu/127.9.44.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
I20260808 23:27:08.687430 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:0
--local_ip_for_outbound_sockets=127.9.44.67
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:40453
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:09.003069 14262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:09.003316 14262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:09.003417 14262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:09.010536 14262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:09.010632 14262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:09.010747 14262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:27:09.020238 14262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:40453
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:09.021306 14262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:09.022548 14262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:09.034699 14267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:09.034788 14269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:09.034785 14271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:09.035519 14262 server_base.cc:1034] running on GCE node
I20260808 23:27:09.036096 14262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:09.036972 14262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:09.038163 14262 hybrid_clock.cc:648] HybridClock initialized: now 1786231629038119 us; error 33 us; skew 500 ppm
I20260808 23:27:09.038468 14262 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:09 dist-test-slave-6bbx krb5kdc[13709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231629, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:09.041477 14262 init.cc:377] Logged in from keytab as kudu/127.9.44.67@KRBTEST.COM (short username kudu)
I20260808 23:27:09.042898 14262 webserver.cc:460] Webserver started at http://127.9.44.67:36967/ using document root <none> and password file <none>
I20260808 23:27:09.043550 14262 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:09.043701 14262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:09.043995 14262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:09.046663 14262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/data/instance:
uuid: "a675d708cef6491c92a17502158aa6ad"
format_stamp: "Formatted at 2026-08-08 23:27:09 on dist-test-slave-6bbx"
server_key: "ca0507b3e88d2416bb3aa0786d722d19"
server_key_iv: "4c2195b8a7e665af532c0ef05d70e6be"
server_key_version: "encryptionkey@0"
I20260808 23:27:09.047398 14262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance:
uuid: "a675d708cef6491c92a17502158aa6ad"
format_stamp: "Formatted at 2026-08-08 23:27:09 on dist-test-slave-6bbx"
server_key: "ca0507b3e88d2416bb3aa0786d722d19"
server_key_iv: "4c2195b8a7e665af532c0ef05d70e6be"
server_key_version: "encryptionkey@0"
I20260808 23:27:09.052013 14262 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 23:27:09.055210 14282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:09.056469 14262 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:27:09.056608 14262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "a675d708cef6491c92a17502158aa6ad"
format_stamp: "Formatted at 2026-08-08 23:27:09 on dist-test-slave-6bbx"
server_key: "ca0507b3e88d2416bb3aa0786d722d19"
server_key_iv: "4c2195b8a7e665af532c0ef05d70e6be"
server_key_version: "encryptionkey@0"
I20260808 23:27:09.056769 14262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:09.098222 14262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:09.102109 14262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:09.102391 14262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:09.103271 14262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:09.104635 14262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:09.104712 14262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:09.104826 14262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:09.104880 14262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:09.134033 14262 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:42539
I20260808 23:27:09.134089 14461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:42539 every 8 connection(s)
I20260808 23:27:09.135164 14262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:27:09.144008 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 14262
I20260808 23:27:09.144168 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance
I20260808 23:27:09.144635 9393 external_mini_cluster.cc:1428] Setting key e02f2d99c2a70e3c91108a5247580733
Aug 08 23:27:09 dist-test-slave-6bbx krb5kdc[13709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231629, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:09.153380 14062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:09.137635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:51613 (local address 127.9.44.126:40453)
0808 23:27:09.137848 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:09.137852 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:09.138981 (+ 1129us) server_negotiation.cc:407] Connection header received
0808 23:27:09.140689 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:09.140693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:09.140762 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:09.140849 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:09.142268 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:09.143029 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:09.144593 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:09.145054 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:09.145186 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:09.148684 (+ 3498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:09.148689 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:09.148692 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:09.148723 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:09.151001 (+ 2278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:09.151611 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:09.151614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:09.151617 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:09.151760 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:09.152094 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:09.152097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:09.152099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:09.152345 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:09.152450 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:09.152989 (+ 539us) server_negotiation.cc:299] Negotiation successful
0808 23:27:09.153147 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 23:27:09.154248 14465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:09.138141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:40453 (local address 127.9.44.67:51613)
0808 23:27:09.138833 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:09.138876 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:09.140458 (+ 1582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:09.141025 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:09.141033 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:09.141501 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:09.142143 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:09.142163 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:09.143158 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:09.143161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:09.144455 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:09.144468 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:09.145260 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:09.145267 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:09.145547 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:09.146197 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:09.146226 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:09.148494 (+ 2268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:09.151113 (+ 2619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:09.151120 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:09.151130 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:09.151525 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:09.151841 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:09.151844 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:09.151846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:09.152014 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:09.152439 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:09.152444 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:09.152699 (+ 255us) client_negotiation.cc:769] Sending connection context
0808 23:27:09.152906 (+ 207us) client_negotiation.cc:240] Negotiation successful
0808 23:27:09.153239 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":282,"threads_started":1}
I20260808 23:27:09.155380 14462 heartbeater.cc:347] Connected to a master server at 127.9.44.126:40453
I20260808 23:27:09.155645 14462 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:09.156325 14462 heartbeater.cc:511] Master 127.9.44.126:40453 requested a full tablet report, sending...
I20260808 23:27:09.157648 13777 ts_manager.cc:194] Registered new tserver with Master: a675d708cef6491c92a17502158aa6ad (127.9.44.67:42539)
I20260808 23:27:09.158463 13777 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.67@KRBTEST.COM'} at 127.9.44.67:51613
I20260808 23:27:09.160060 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 23:27:09 dist-test-slave-6bbx krb5kdc[13709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231629, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
I20260808 23:27:09.198457 14059 heartbeater.cc:503] Master 127.9.44.126:40453 was elected leader, sending a full tablet report...
Aug 08 23:27:09 dist-test-slave-6bbx krb5kdc[13709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231629, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:09.208822 14062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:09.195820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58944 (local address 127.9.44.126:40453)
0808 23:27:09.195987 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:09.195991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:09.196451 (+ 460us) server_negotiation.cc:407] Connection header received
0808 23:27:09.196622 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:09.196626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:09.196688 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:09.196780 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:09.198077 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:09.198985 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:09.200317 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:09.200748 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:09.200863 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:09.204440 (+ 3577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:09.204445 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:09.204448 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:09.204480 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:09.206663 (+ 2183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:09.207240 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:09.207243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:09.207246 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:09.207363 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:09.207693 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:09.207698 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:09.207701 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:09.208024 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:09.208203 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:09.208449 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 23:27:09.208604 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 23:27:09.209332 13776 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58944
I20260808 23:27:09.211014 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 13870
Aug 08 23:27:09 dist-test-slave-6bbx krb5kdc[13709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231629, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.44.65@KRBTEST.COM
I20260808 23:27:09.225363 14474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:09.211901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41980 (local address 127.9.44.65:35357)
0808 23:27:09.212474 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:09.212481 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:09.212504 (+ 23us) server_negotiation.cc:407] Connection header received
0808 23:27:09.212553 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:09.212560 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:09.212765 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:09.212902 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:09.214035 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:09.215087 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:09.216324 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:09.216865 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:09.216989 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:09.220555 (+ 3566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:09.220566 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:09.220578 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:09.220609 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:09.223055 (+ 2446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:09.223608 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:09.223626 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:09.223628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:09.223789 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:09.224079 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:09.224081 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:09.224083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:09.224400 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:09.224636 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:09.224781 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 23:27:09.224964 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":239,"threads_started":1}
W20260808 23:27:09.226519 14053 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41980
W20260808 23:27:09.227523 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 13870: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:27:09.227730 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 13870
I20260808 23:27:09.253304 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 14068
Aug 08 23:27:09 dist-test-slave-6bbx krb5kdc[13709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231629, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.44.66@KRBTEST.COM
I20260808 23:27:09.268386 14475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:09.254089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42836 (local address 127.9.44.66:39623)
0808 23:27:09.254578 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:09.254585 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:09.254609 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:27:09.254680 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:09.254686 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:09.254888 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:09.255035 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:09.256325 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:09.257328 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:09.258522 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:09.259061 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:09.259179 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:09.263072 (+ 3893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:09.263086 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:09.263103 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:09.263143 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:09.265781 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:09.266372 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:09.266379 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:09.266381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:09.266501 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:09.266846 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:09.266851 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:09.266854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:09.267178 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:09.267474 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:09.267807 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 23:27:09.268063 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":251,"threads_started":1}
W20260808 23:27:09.269610 14244 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42836
W20260808 23:27:09.270649 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 14068: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:27:09.270779 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 14068
I20260808 23:27:09.296896 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 14262
Aug 08 23:27:09 dist-test-slave-6bbx krb5kdc[13709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231629, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.44.67@KRBTEST.COM
I20260808 23:27:09.311650 14476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:09.297854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43060 (local address 127.9.44.67:42539)
0808 23:27:09.298469 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:09.298476 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:09.298504 (+ 28us) server_negotiation.cc:407] Connection header received
0808 23:27:09.298561 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:09.298567 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:09.298761 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:09.298890 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:09.300104 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:09.301124 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:09.302291 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:09.302812 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:09.302910 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:09.306816 (+ 3906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:09.306826 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:09.306839 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:09.306882 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:09.309498 (+ 2616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:09.310042 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:09.310048 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:09.310050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:09.310163 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:09.310413 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:09.310416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:09.310418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:09.310717 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:09.310950 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:09.311119 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 23:27:09.311308 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":298,"threads_started":1}
W20260808 23:27:09.312952 14454 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43060
W20260808 23:27:09.313952 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 14262: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:27:09.314090 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 14262
I20260808 23:27:09.347735 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 13728
I20260808 23:27:09.348781 13823 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:09.449467 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 13728
2026-08-08T23:27:09Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3604 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:27:09 dist-test-slave-6bbx krb5kdc[14483](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:27:09 dist-test-slave-6bbx krb5kdc[14483](info): set up 2 sockets
Aug 08 23:27:09 dist-test-slave-6bbx krb5kdc[14483](info): commencing operation
krb5kdc: starting...
W20260808 23:27:10.614167 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:27:10 dist-test-slave-6bbx krb5kdc[14483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:27:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:27:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:27:10.798838 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:33333
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:33333
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:11.128400 14504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:11.128677 14504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:11.128811 14504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:11.135890 14504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:27:11.135986 14504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:11.136058 14504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:11.136142 14504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:27:11.136196 14504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:27:11.145572 14504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:33333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:33333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:11.146716 14504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:11.148020 14504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:11.159160 14513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:11.159237 14510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:11.159211 14511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:11.160122 14504 server_base.cc:1034] running on GCE node
I20260808 23:27:11.160815 14504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:11.162068 14504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:11.163290 14504 hybrid_clock.cc:648] HybridClock initialized: now 1786231631163239 us; error 36 us; skew 500 ppm
I20260808 23:27:11.163594 14504 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:11 dist-test-slave-6bbx krb5kdc[14483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231631, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:11.166491 14504 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:27:11.167887 14504 webserver.cc:460] Webserver started at http://127.9.44.126:34525/ using document root <none> and password file <none>
I20260808 23:27:11.168573 14504 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:11.168676 14504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:11.168942 14504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:11.171720 14504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "57df25a6f33a4e6b89c0d8feaf0b3f4c"
format_stamp: "Formatted at 2026-08-08 23:27:11 on dist-test-slave-6bbx"
server_key: "bf336deb87bb4d6c92960766f2125b29"
server_key_iv: "a4f02a0b12f8021546ae66944b4fdf74"
server_key_version: "encryptionkey@0"
I20260808 23:27:11.172459 14504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "57df25a6f33a4e6b89c0d8feaf0b3f4c"
format_stamp: "Formatted at 2026-08-08 23:27:11 on dist-test-slave-6bbx"
server_key: "bf336deb87bb4d6c92960766f2125b29"
server_key_iv: "a4f02a0b12f8021546ae66944b4fdf74"
server_key_version: "encryptionkey@0"
I20260808 23:27:11.177419 14504 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260808 23:27:11.180619 14526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:11.181802 14504 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 23:27:11.181964 14504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "57df25a6f33a4e6b89c0d8feaf0b3f4c"
format_stamp: "Formatted at 2026-08-08 23:27:11 on dist-test-slave-6bbx"
server_key: "bf336deb87bb4d6c92960766f2125b29"
server_key_iv: "a4f02a0b12f8021546ae66944b4fdf74"
server_key_version: "encryptionkey@0"
I20260808 23:27:11.182153 14504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:11.199057 14504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:11.202622 14504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:11.202876 14504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:11.219990 14504 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:33333
I20260808 23:27:11.220023 14609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:33333 every 8 connection(s)
I20260808 23:27:11.221230 14504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:27:11.224299 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 14504
I20260808 23:27:11.224521 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:27:11.224973 14610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:11.225303 9393 external_mini_cluster.cc:1428] Setting key 951947c1ad916746b8bc2d4cd8387103
I20260808 23:27:11.236953 14610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57df25a6f33a4e6b89c0d8feaf0b3f4c: Bootstrap starting.
Aug 08 23:27:11 dist-test-slave-6bbx krb5kdc[14483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:11.240147 14610 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57df25a6f33a4e6b89c0d8feaf0b3f4c: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:11.241290 14610 log.cc:824] T 00000000000000000000000000000000 P 57df25a6f33a4e6b89c0d8feaf0b3f4c: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:11.244426 14610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57df25a6f33a4e6b89c0d8feaf0b3f4c: No bootstrap required, opened a new log
I20260808 23:27:11.246670 14614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:11.227293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38668 (local address 127.9.44.126:33333)
0808 23:27:11.228157 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:11.228168 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:11.228210 (+ 42us) server_negotiation.cc:407] Connection header received
0808 23:27:11.229745 (+ 1535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:11.229766 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:11.230129 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:11.230481 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:11.231986 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:11.233238 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:11.234488 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:11.235188 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:11.235359 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:11.239359 (+ 4000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:11.239366 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:11.239380 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:11.239414 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:11.242769 (+ 3355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:11.243552 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:11.243559 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:11.243564 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:11.243778 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:11.244168 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:11.244171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:11.244173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:11.244705 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:11.244941 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:11.245354 (+ 413us) server_negotiation.cc:299] Negotiation successful
0808 23:27:11.245677 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":378,"threads_started":1}
I20260808 23:27:11.252636 14610 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57df25a6f33a4e6b89c0d8feaf0b3f4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57df25a6f33a4e6b89c0d8feaf0b3f4c" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 33333 } }
I20260808 23:27:11.252974 14610 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57df25a6f33a4e6b89c0d8feaf0b3f4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:11.253060 14610 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57df25a6f33a4e6b89c0d8feaf0b3f4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57df25a6f33a4e6b89c0d8feaf0b3f4c, State: Initialized, Role: FOLLOWER
I20260808 23:27:11.253626 14610 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57df25a6f33a4e6b89c0d8feaf0b3f4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57df25a6f33a4e6b89c0d8feaf0b3f4c" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 33333 } }
I20260808 23:27:11.253916 14610 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57df25a6f33a4e6b89c0d8feaf0b3f4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:27:11.254041 14610 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57df25a6f33a4e6b89c0d8feaf0b3f4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:27:11.254194 14610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57df25a6f33a4e6b89c0d8feaf0b3f4c [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:11.256492 14610 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57df25a6f33a4e6b89c0d8feaf0b3f4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57df25a6f33a4e6b89c0d8feaf0b3f4c" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 33333 } }
I20260808 23:27:11.257031 14610 leader_election.cc:302] T 00000000000000000000000000000000 P 57df25a6f33a4e6b89c0d8feaf0b3f4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57df25a6f33a4e6b89c0d8feaf0b3f4c; no voters:
I20260808 23:27:11.257516 14610 leader_election.cc:288] T 00000000000000000000000000000000 P 57df25a6f33a4e6b89c0d8feaf0b3f4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:27:11.257630 14618 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57df25a6f33a4e6b89c0d8feaf0b3f4c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:27:11.258267 14618 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57df25a6f33a4e6b89c0d8feaf0b3f4c [term 1 LEADER]: Becoming Leader. State: Replica: 57df25a6f33a4e6b89c0d8feaf0b3f4c, State: Running, Role: LEADER
I20260808 23:27:11.258759 14618 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57df25a6f33a4e6b89c0d8feaf0b3f4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57df25a6f33a4e6b89c0d8feaf0b3f4c" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 33333 } }
I20260808 23:27:11.259337 14610 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57df25a6f33a4e6b89c0d8feaf0b3f4c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:27:11.261900 14620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57df25a6f33a4e6b89c0d8feaf0b3f4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57df25a6f33a4e6b89c0d8feaf0b3f4c. Latest consensus state: current_term: 1 leader_uuid: "57df25a6f33a4e6b89c0d8feaf0b3f4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57df25a6f33a4e6b89c0d8feaf0b3f4c" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 33333 } } }
I20260808 23:27:11.262225 14620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57df25a6f33a4e6b89c0d8feaf0b3f4c [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:11.262889 14619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57df25a6f33a4e6b89c0d8feaf0b3f4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57df25a6f33a4e6b89c0d8feaf0b3f4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57df25a6f33a4e6b89c0d8feaf0b3f4c" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 33333 } } }
I20260808 23:27:11.263252 14619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57df25a6f33a4e6b89c0d8feaf0b3f4c [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:11.263489 14629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:27:11.268421 14629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:27:11.276748 14629 catalog_manager.cc:1347] Generated new cluster ID: 67a1a9ccaeb34ed8828d0a441c4613ef
I20260808 23:27:11.276841 14629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:27:11.296345 14629 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:27:11.297495 14629 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:27:11.303414 14629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57df25a6f33a4e6b89c0d8feaf0b3f4c: Generated new TSK 0
I20260808 23:27:11.304214 14629 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.44.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
I20260808 23:27:11.361163 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:0
--local_ip_for_outbound_sockets=127.9.44.65
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:33333
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:11.680691 14647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:11.680961 14647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:11.681099 14647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:11.688194 14647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:11.688304 14647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:11.688441 14647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:27:11.697820 14647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:33333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:11.698930 14647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:11.700201 14647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:11.712438 14662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:11.712502 14661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:11.712528 14664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:11.713557 14647 server_base.cc:1034] running on GCE node
I20260808 23:27:11.714151 14647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:11.715008 14647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:11.716202 14647 hybrid_clock.cc:648] HybridClock initialized: now 1786231631716139 us; error 51 us; skew 500 ppm
I20260808 23:27:11.716502 14647 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:11 dist-test-slave-6bbx krb5kdc[14483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231631, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:11.719462 14647 init.cc:377] Logged in from keytab as kudu/127.9.44.65@KRBTEST.COM (short username kudu)
I20260808 23:27:11.720798 14647 webserver.cc:460] Webserver started at http://127.9.44.65:45175/ using document root <none> and password file <none>
I20260808 23:27:11.721431 14647 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:11.721549 14647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:11.721832 14647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:11.724673 14647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-0/data/instance:
uuid: "abadbb4f64a542ccb627407a1b3e2516"
format_stamp: "Formatted at 2026-08-08 23:27:11 on dist-test-slave-6bbx"
server_key: "3045c6a732c42d19f08fb98b27ce5945"
server_key_iv: "550564a976a5ac31ff8480b4407b9902"
server_key_version: "encryptionkey@0"
I20260808 23:27:11.725423 14647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance:
uuid: "abadbb4f64a542ccb627407a1b3e2516"
format_stamp: "Formatted at 2026-08-08 23:27:11 on dist-test-slave-6bbx"
server_key: "3045c6a732c42d19f08fb98b27ce5945"
server_key_iv: "550564a976a5ac31ff8480b4407b9902"
server_key_version: "encryptionkey@0"
I20260808 23:27:11.730508 14647 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 23:27:11.733842 14677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:11.735069 14647 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260808 23:27:11.735221 14647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "abadbb4f64a542ccb627407a1b3e2516"
format_stamp: "Formatted at 2026-08-08 23:27:11 on dist-test-slave-6bbx"
server_key: "3045c6a732c42d19f08fb98b27ce5945"
server_key_iv: "550564a976a5ac31ff8480b4407b9902"
server_key_version: "encryptionkey@0"
I20260808 23:27:11.735411 14647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:11.745363 14647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:11.749136 14647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:11.749390 14647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:11.750239 14647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:11.751441 14647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:11.751518 14647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:11.751629 14647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:11.751719 14647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260808 23:27:11.781337 14647 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:45117
I20260808 23:27:11.781381 14844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:45117 every 8 connection(s)
I20260808 23:27:11.782471 14647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:27:11.788430 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 14647
I20260808 23:27:11.788617 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance
I20260808 23:27:11.789224 9393 external_mini_cluster.cc:1428] Setting key 1a6fec8d18ee0733daa593a10de4736f
Aug 08 23:27:11 dist-test-slave-6bbx krb5kdc[14483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231631, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:11.800947 14849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:11.784729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:53871 (local address 127.9.44.126:33333)
0808 23:27:11.785142 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:11.785146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:11.786013 (+ 867us) server_negotiation.cc:407] Connection header received
0808 23:27:11.787871 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:11.787878 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:11.787962 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:11.788059 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:11.789579 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:11.790472 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:11.792136 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:11.792541 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:11.792647 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:11.795753 (+ 3106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:11.795757 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:11.795762 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:11.795795 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:11.798063 (+ 2268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:11.798881 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:11.798885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:11.798887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:11.799011 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:11.799438 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:11.799441 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:11.799443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:11.799749 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:11.799858 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:11.800540 (+ 682us) server_negotiation.cc:299] Negotiation successful
0808 23:27:11.800695 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":217,"threads_started":1}
I20260808 23:27:11.802299 14853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:11.785115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:33333 (local address 127.9.44.65:53871)
0808 23:27:11.785844 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:11.785899 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:11.787580 (+ 1681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:11.788313 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:11.788321 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:11.788852 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:11.789415 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:11.789435 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:11.790616 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:11.790620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:11.791981 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:11.791994 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:11.792756 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:11.792764 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:11.792982 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:11.793518 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:11.793538 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:11.795538 (+ 2000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:11.798192 (+ 2654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:11.798201 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:11.798215 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:11.798738 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:11.799108 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:11.799112 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:11.799115 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:11.799343 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:11.799871 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:11.799878 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:11.800217 (+ 339us) client_negotiation.cc:769] Sending connection context
0808 23:27:11.800492 (+ 275us) client_negotiation.cc:240] Negotiation successful
0808 23:27:11.800959 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":294,"threads_started":1}
I20260808 23:27:11.803785 14847 heartbeater.cc:347] Connected to a master server at 127.9.44.126:33333
I20260808 23:27:11.804055 14847 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:11.804677 14847 heartbeater.cc:511] Master 127.9.44.126:33333 requested a full tablet report, sending...
I20260808 23:27:11.806565 14550 ts_manager.cc:194] Registered new tserver with Master: abadbb4f64a542ccb627407a1b3e2516 (127.9.44.65:45117)
I20260808 23:27:11.808043 14550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.65@KRBTEST.COM'} at 127.9.44.65:53871
WARNING: no policy specified for kudu/127.9.44.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
I20260808 23:27:11.834028 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:0
--local_ip_for_outbound_sockets=127.9.44.66
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:33333
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:12.145627 14856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:12.145933 14856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:12.146081 14856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:12.152887 14856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:12.152990 14856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:12.153142 14856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:27:12.162269 14856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:33333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:12.163348 14856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:12.164567 14856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:12.176398 14862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:12.176407 14865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:12.176427 14863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:12.177493 14856 server_base.cc:1034] running on GCE node
I20260808 23:27:12.178035 14856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:12.178848 14856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:12.180030 14856 hybrid_clock.cc:648] HybridClock initialized: now 1786231632179970 us; error 50 us; skew 500 ppm
I20260808 23:27:12.180333 14856 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:12 dist-test-slave-6bbx krb5kdc[14483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231632, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:12.183557 14856 init.cc:377] Logged in from keytab as kudu/127.9.44.66@KRBTEST.COM (short username kudu)
I20260808 23:27:12.184841 14856 webserver.cc:460] Webserver started at http://127.9.44.66:33813/ using document root <none> and password file <none>
I20260808 23:27:12.185449 14856 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:12.185587 14856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:12.185880 14856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:12.188663 14856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-1/data/instance:
uuid: "42f0547d45fb4af79fa46a09ed21f305"
format_stamp: "Formatted at 2026-08-08 23:27:12 on dist-test-slave-6bbx"
server_key: "8c77e31ab85c93f246a5f817fe9e1831"
server_key_iv: "1976443ad0dd55bc0e727663fe4b2c49"
server_key_version: "encryptionkey@0"
I20260808 23:27:12.189381 14856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance:
uuid: "42f0547d45fb4af79fa46a09ed21f305"
format_stamp: "Formatted at 2026-08-08 23:27:12 on dist-test-slave-6bbx"
server_key: "8c77e31ab85c93f246a5f817fe9e1831"
server_key_iv: "1976443ad0dd55bc0e727663fe4b2c49"
server_key_version: "encryptionkey@0"
I20260808 23:27:12.194064 14856 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 23:27:12.197280 14874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:12.198454 14856 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 23:27:12.198587 14856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "42f0547d45fb4af79fa46a09ed21f305"
format_stamp: "Formatted at 2026-08-08 23:27:12 on dist-test-slave-6bbx"
server_key: "8c77e31ab85c93f246a5f817fe9e1831"
server_key_iv: "1976443ad0dd55bc0e727663fe4b2c49"
server_key_version: "encryptionkey@0"
I20260808 23:27:12.198765 14856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:12.209431 14856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:12.212826 14856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:12.213068 14856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:12.213892 14856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:12.215058 14856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:12.215127 14856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:12.215245 14856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:12.215294 14856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:12.244831 14856 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:43251
I20260808 23:27:12.244875 15034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:43251 every 8 connection(s)
I20260808 23:27:12.245931 14856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:27:12.249293 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 14856
I20260808 23:27:12.249491 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance
I20260808 23:27:12.250007 9393 external_mini_cluster.cc:1428] Setting key a65dc9309276b9d86c8fd23dd4b4321b
Aug 08 23:27:12 dist-test-slave-6bbx krb5kdc[14483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231632, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:12.265951 14849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:12.248231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:59689 (local address 127.9.44.126:33333)
0808 23:27:12.248393 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:12.248397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:12.249588 (+ 1191us) server_negotiation.cc:407] Connection header received
0808 23:27:12.251834 (+ 2246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:12.251840 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:12.251923 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:12.252015 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:12.253878 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:12.254587 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:12.256586 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:12.256978 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:12.257066 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:12.260714 (+ 3648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:12.260719 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:12.260722 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:12.260755 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:12.263160 (+ 2405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:12.263833 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:12.263837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:12.263839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:12.263971 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:12.264379 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:12.264382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:12.264384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:12.264636 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:12.264774 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:12.265417 (+ 643us) server_negotiation.cc:299] Negotiation successful
0808 23:27:12.265635 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 23:27:12.266798 15037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:12.248636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:33333 (local address 127.9.44.66:59689)
0808 23:27:12.249433 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:12.249483 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:12.251519 (+ 2036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:12.252288 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:12.252299 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:12.252978 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:12.253725 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:12.253750 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:12.254738 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:12.254744 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:12.256460 (+ 1716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:12.256482 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:12.257177 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:12.257183 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:12.257455 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:12.258082 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:12.258108 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:12.260515 (+ 2407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:12.263285 (+ 2770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:12.263294 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:12.263308 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:12.263717 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:12.264073 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:12.264078 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:12.264082 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:12.264266 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:12.264766 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:12.264773 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:12.265060 (+ 287us) client_negotiation.cc:769] Sending connection context
0808 23:27:12.265286 (+ 226us) client_negotiation.cc:240] Negotiation successful
0808 23:27:12.265733 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":295,"threads_started":1}
I20260808 23:27:12.268081 15035 heartbeater.cc:347] Connected to a master server at 127.9.44.126:33333
I20260808 23:27:12.268410 15035 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:12.269245 15035 heartbeater.cc:511] Master 127.9.44.126:33333 requested a full tablet report, sending...
I20260808 23:27:12.270725 14550 ts_manager.cc:194] Registered new tserver with Master: 42f0547d45fb4af79fa46a09ed21f305 (127.9.44.66:43251)
I20260808 23:27:12.271485 14550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.66@KRBTEST.COM'} at 127.9.44.66:59689
WARNING: no policy specified for kudu/127.9.44.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
I20260808 23:27:12.294529 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:0
--local_ip_for_outbound_sockets=127.9.44.67
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:33333
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:39677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:12.604046 15042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:12.604311 15042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:12.604449 15042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:12.611593 15042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:12.611719 15042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:12.611876 15042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:27:12.620982 15042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:39677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:33333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:12.622090 15042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:12.623303 15042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:12.635197 15058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:12.635275 15054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:12.635241 15055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:12.636214 15042 server_base.cc:1034] running on GCE node
I20260808 23:27:12.636780 15042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:12.637598 15042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:12.638793 15042 hybrid_clock.cc:648] HybridClock initialized: now 1786231632638746 us; error 47 us; skew 500 ppm
I20260808 23:27:12.639065 15042 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:12 dist-test-slave-6bbx krb5kdc[14483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231632, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:12.641952 15042 init.cc:377] Logged in from keytab as kudu/127.9.44.67@KRBTEST.COM (short username kudu)
I20260808 23:27:12.643179 15042 webserver.cc:460] Webserver started at http://127.9.44.67:42583/ using document root <none> and password file <none>
I20260808 23:27:12.643814 15042 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:12.643935 15042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:12.644217 15042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:12.646899 15042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-2/data/instance:
uuid: "2dc22b2135574fe28c12be2f934bd42e"
format_stamp: "Formatted at 2026-08-08 23:27:12 on dist-test-slave-6bbx"
server_key: "3e1bb1941d709de85f4c82c92bd29f18"
server_key_iv: "97f0696cd535accf7eee3a86a3fff006"
server_key_version: "encryptionkey@0"
I20260808 23:27:12.647599 15042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance:
uuid: "2dc22b2135574fe28c12be2f934bd42e"
format_stamp: "Formatted at 2026-08-08 23:27:12 on dist-test-slave-6bbx"
server_key: "3e1bb1941d709de85f4c82c92bd29f18"
server_key_iv: "97f0696cd535accf7eee3a86a3fff006"
server_key_version: "encryptionkey@0"
I20260808 23:27:12.652202 15042 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 23:27:12.655303 15066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:12.656489 15042 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 23:27:12.656633 15042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "2dc22b2135574fe28c12be2f934bd42e"
format_stamp: "Formatted at 2026-08-08 23:27:12 on dist-test-slave-6bbx"
server_key: "3e1bb1941d709de85f4c82c92bd29f18"
server_key_iv: "97f0696cd535accf7eee3a86a3fff006"
server_key_version: "encryptionkey@0"
I20260808 23:27:12.656812 15042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:12.686609 15042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:12.690255 15042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:12.690500 15042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:12.691499 15042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:12.692839 15042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:12.692920 15042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:12.693037 15042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:12.693125 15042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:12.722870 15042 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:45213
I20260808 23:27:12.722932 15241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:45213 every 8 connection(s)
I20260808 23:27:12.724088 15042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:27:12.729912 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 15042
I20260808 23:27:12.730083 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance
I20260808 23:27:12.730486 9393 external_mini_cluster.cc:1428] Setting key 14319bbe375ab7c27566a8e301f8b532
Aug 08 23:27:12 dist-test-slave-6bbx krb5kdc[14483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231632, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:12.742753 14849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:12.726394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:39387 (local address 127.9.44.126:33333)
0808 23:27:12.726563 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:12.726568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:12.727689 (+ 1121us) server_negotiation.cc:407] Connection header received
0808 23:27:12.729491 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:12.729497 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:12.729588 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:12.729684 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:12.731296 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:12.732331 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:12.733824 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:12.734233 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:12.734340 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:12.737471 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:12.737478 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:12.737481 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:12.737514 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:12.739965 (+ 2451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:12.740610 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:12.740624 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:12.740628 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:12.740749 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:12.741142 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:12.741148 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:12.741151 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:12.741425 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:12.741556 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:12.742207 (+ 651us) server_negotiation.cc:299] Negotiation successful
0808 23:27:12.742422 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 23:27:12.743580 15245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:12.726806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:33333 (local address 127.9.44.67:39387)
0808 23:27:12.727503 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:12.727543 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:12.729229 (+ 1686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:12.729929 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:12.729937 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:12.730484 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:12.731100 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:12.731141 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:12.732480 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:12.732484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:12.733687 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:12.733700 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:12.734447 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:12.734455 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:12.734678 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:12.735231 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:12.735252 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:12.737271 (+ 2019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:12.740087 (+ 2816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:12.740096 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:12.740110 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:12.740497 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:12.740852 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:12.740857 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:12.740861 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:12.741034 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:12.741549 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:12.741556 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:12.741841 (+ 285us) client_negotiation.cc:769] Sending connection context
0808 23:27:12.742078 (+ 237us) client_negotiation.cc:240] Negotiation successful
0808 23:27:12.742549 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":297,"threads_started":1}
I20260808 23:27:12.744966 15242 heartbeater.cc:347] Connected to a master server at 127.9.44.126:33333
I20260808 23:27:12.745283 15242 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:12.745963 15242 heartbeater.cc:511] Master 127.9.44.126:33333 requested a full tablet report, sending...
I20260808 23:27:12.747247 14550 ts_manager.cc:194] Registered new tserver with Master: 2dc22b2135574fe28c12be2f934bd42e (127.9.44.67:45213)
I20260808 23:27:12.748062 14550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.67@KRBTEST.COM'} at 127.9.44.67:39387
I20260808 23:27:12.757314 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 23:27:12 dist-test-slave-6bbx krb5kdc[14483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 23:27:12 dist-test-slave-6bbx krb5kdc[14483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.65@KRBTEST.COM
I20260808 23:27:12.810636 14847 heartbeater.cc:503] Master 127.9.44.126:33333 was elected leader, sending a full tablet report...
I20260808 23:27:12.813560 15250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:12.794797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59426 (local address 127.9.44.65:45117)
0808 23:27:12.795390 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:12.795397 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:12.795423 (+ 26us) server_negotiation.cc:407] Connection header received
0808 23:27:12.795478 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:12.795484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:12.795703 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:12.795828 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:12.797084 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:12.798053 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:12.799202 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:12.799836 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:12.799951 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:12.804916 (+ 4965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:12.804929 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:12.804946 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:12.804985 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:12.810734 (+ 5749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:12.811524 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:12.811533 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:12.811536 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:12.811717 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:12.812095 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:12.812100 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:12.812103 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:12.812543 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:12.812761 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:12.813002 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 23:27:12.813194 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":324,"threads_started":1}
Aug 08 23:27:12 dist-test-slave-6bbx krb5kdc[14483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:12.829458 14849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:12.817484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38676 (local address 127.9.44.126:33333)
0808 23:27:12.817687 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:12.817692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:12.817841 (+ 149us) server_negotiation.cc:407] Connection header received
0808 23:27:12.817988 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:12.817992 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:12.818052 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:12.818120 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:12.819188 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:12.819992 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:12.821204 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:12.821614 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:12.821701 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:12.825103 (+ 3402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:12.825108 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:12.825111 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:12.825143 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:12.827341 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:12.827939 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:12.827942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:12.827945 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:12.828064 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:12.828329 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:12.828332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:12.828334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:12.828623 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:12.828807 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:12.829027 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 23:27:12.829175 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260808 23:27:12.830123 14550 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38676
I20260808 23:27:12.831029 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 14647
I20260808 23:27:12.841953 15250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:12.831788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59438 (local address 127.9.44.65:45117)
0808 23:27:12.831922 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:12.831926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:12.832215 (+ 289us) server_negotiation.cc:407] Connection header received
0808 23:27:12.832318 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:12.832321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:12.832375 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:12.832448 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:12.833481 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:12.834097 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:12.835180 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:12.835568 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:12.835658 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:12.837594 (+ 1936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:12.837598 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:12.837601 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:12.837629 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:12.840052 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:12.840569 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:12.840572 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:12.840574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:12.840680 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:12.840952 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:12.840955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:12.840957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:12.841163 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:12.841341 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:12.841586 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 23:27:12.841743 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 23:27:12.842459 14839 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:12.922802 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 14647
I20260808 23:27:12.943511 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 14856
Aug 08 23:27:12 dist-test-slave-6bbx krb5kdc[14483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.66@KRBTEST.COM
I20260808 23:27:12.958640 15259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:12.944463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53372 (local address 127.9.44.66:43251)
0808 23:27:12.944979 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:12.944986 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:12.945012 (+ 26us) server_negotiation.cc:407] Connection header received
0808 23:27:12.945075 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:12.945081 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:12.945301 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:12.945446 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:12.946713 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:12.947756 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:12.949020 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:12.949564 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:12.949700 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:12.953817 (+ 4117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:12.953829 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:12.953844 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:12.953888 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:12.956349 (+ 2461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:12.956911 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:12.956918 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:12.956920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:12.957032 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:12.957318 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:12.957321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:12.957323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:12.957679 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:12.958001 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:12.958172 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 23:27:12.958364 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":240,"threads_started":1}
I20260808 23:27:12.960059 15027 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:13.040678 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 14856
I20260808 23:27:13.062084 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 15042
Aug 08 23:27:13 dist-test-slave-6bbx krb5kdc[14483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.67@KRBTEST.COM
I20260808 23:27:13.079367 15262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:13.063077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60470 (local address 127.9.44.67:45213)
0808 23:27:13.063788 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:13.063795 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:13.063825 (+ 30us) server_negotiation.cc:407] Connection header received
0808 23:27:13.063901 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:13.063908 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:13.064141 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:13.064268 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:13.065542 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:13.066594 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:13.067953 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:13.068501 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:13.068600 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:13.072755 (+ 4155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:13.072767 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:13.072782 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:13.072815 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:13.076256 (+ 3441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:13.077038 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:13.077048 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:13.077052 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:13.077222 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:13.077587 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:13.077592 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:13.077596 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:13.078062 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:13.078394 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:13.078713 (+ 319us) server_negotiation.cc:299] Negotiation successful
0808 23:27:13.078979 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":357,"threads_started":1}
I20260808 23:27:13.080920 15234 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:13.160725 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 15042
I20260808 23:27:13.180533 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 14504
I20260808 23:27:13.181630 14603 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:13.281824 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 14504
2026-08-08T23:27:13Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3839 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:27:13 dist-test-slave-6bbx krb5kdc[15269](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:27:13 dist-test-slave-6bbx krb5kdc[15269](info): set up 2 sockets
Aug 08 23:27:13 dist-test-slave-6bbx krb5kdc[15269](info): commencing operation
krb5kdc: starting...
W20260808 23:27:14.457386 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:27:14 dist-test-slave-6bbx krb5kdc[15269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:27:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:27:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:27:14.644330 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:36003
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:36003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:14.952121 15288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:14.952407 15288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:14.952557 15288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:14.959494 15288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:27:14.959607 15288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:14.959705 15288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:14.959792 15288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:27:14.959867 15288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:27:14.969092 15288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:36003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:36003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:14.970196 15288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:14.971388 15288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:14.982197 15296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:14.982241 15297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:14.982162 15301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:14.983259 15288 server_base.cc:1034] running on GCE node
I20260808 23:27:14.984002 15288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:14.985229 15288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:14.986428 15288 hybrid_clock.cc:648] HybridClock initialized: now 1786231634986388 us; error 42 us; skew 500 ppm
I20260808 23:27:14.986740 15288 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:14 dist-test-slave-6bbx krb5kdc[15269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231634, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:14.989642 15288 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:27:14.990737 15288 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:14.990839 15288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:14.991129 15288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:14.993952 15288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "9954b7c52019417da7fad83c7b23e589"
format_stamp: "Formatted at 2026-08-08 23:27:14 on dist-test-slave-6bbx"
server_key: "663d9535d5a2ed0ee238494ab4bc9c88"
server_key_iv: "6a462254fec415e30a7b667b31245136"
server_key_version: "encryptionkey@0"
I20260808 23:27:14.994716 15288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "9954b7c52019417da7fad83c7b23e589"
format_stamp: "Formatted at 2026-08-08 23:27:14 on dist-test-slave-6bbx"
server_key: "663d9535d5a2ed0ee238494ab4bc9c88"
server_key_iv: "6a462254fec415e30a7b667b31245136"
server_key_version: "encryptionkey@0"
I20260808 23:27:14.999390 15288 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260808 23:27:15.002586 15307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:15.003806 15288 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 23:27:15.003965 15288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "9954b7c52019417da7fad83c7b23e589"
format_stamp: "Formatted at 2026-08-08 23:27:14 on dist-test-slave-6bbx"
server_key: "663d9535d5a2ed0ee238494ab4bc9c88"
server_key_iv: "6a462254fec415e30a7b667b31245136"
server_key_version: "encryptionkey@0"
I20260808 23:27:15.004163 15288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:15.024922 15288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:15.028414 15288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:15.028641 15288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:15.045497 15288 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:36003
I20260808 23:27:15.045519 15383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:36003 every 8 connection(s)
I20260808 23:27:15.046314 15288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:27:15.048851 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 15288
I20260808 23:27:15.049089 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:27:15.049873 9393 external_mini_cluster.cc:1428] Setting key 4c17bf1fff88c724c81263609e96b6a2
I20260808 23:27:15.050321 15386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:15.064236 15386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9954b7c52019417da7fad83c7b23e589: Bootstrap starting.
Aug 08 23:27:15 dist-test-slave-6bbx krb5kdc[15269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:15.067346 15386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9954b7c52019417da7fad83c7b23e589: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:15.068437 15386 log.cc:824] T 00000000000000000000000000000000 P 9954b7c52019417da7fad83c7b23e589: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:15.071019 15386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9954b7c52019417da7fad83c7b23e589: No bootstrap required, opened a new log
I20260808 23:27:15.072201 15391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:15.051827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52530 (local address 127.9.44.126:36003)
0808 23:27:15.052578 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:15.052588 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:15.052627 (+ 39us) server_negotiation.cc:407] Connection header received
0808 23:27:15.054806 (+ 2179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:15.054827 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:15.055168 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:15.055488 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:15.057267 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:15.058518 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:15.060280 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:15.060945 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:15.061076 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:15.065577 (+ 4501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:15.065585 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:15.065598 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:15.065634 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:15.068476 (+ 2842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:15.069096 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:15.069103 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:15.069112 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:15.069283 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:15.069654 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:15.069658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:15.069660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:15.070141 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:15.070419 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:15.070841 (+ 422us) server_negotiation.cc:299] Negotiation successful
0808 23:27:15.071208 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":366,"threads_started":1}
I20260808 23:27:15.080384 15386 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9954b7c52019417da7fad83c7b23e589 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9954b7c52019417da7fad83c7b23e589" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 36003 } }
I20260808 23:27:15.080830 15386 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9954b7c52019417da7fad83c7b23e589 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:15.080955 15386 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9954b7c52019417da7fad83c7b23e589 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9954b7c52019417da7fad83c7b23e589, State: Initialized, Role: FOLLOWER
I20260808 23:27:15.081588 15386 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9954b7c52019417da7fad83c7b23e589 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9954b7c52019417da7fad83c7b23e589" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 36003 } }
I20260808 23:27:15.081887 15386 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9954b7c52019417da7fad83c7b23e589 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:27:15.082036 15386 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9954b7c52019417da7fad83c7b23e589 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:27:15.082227 15386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9954b7c52019417da7fad83c7b23e589 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:15.084510 15386 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9954b7c52019417da7fad83c7b23e589 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9954b7c52019417da7fad83c7b23e589" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 36003 } }
I20260808 23:27:15.085001 15386 leader_election.cc:302] T 00000000000000000000000000000000 P 9954b7c52019417da7fad83c7b23e589 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9954b7c52019417da7fad83c7b23e589; no voters:
I20260808 23:27:15.085538 15386 leader_election.cc:288] T 00000000000000000000000000000000 P 9954b7c52019417da7fad83c7b23e589 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:27:15.085661 15393 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9954b7c52019417da7fad83c7b23e589 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:27:15.086280 15393 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9954b7c52019417da7fad83c7b23e589 [term 1 LEADER]: Becoming Leader. State: Replica: 9954b7c52019417da7fad83c7b23e589, State: Running, Role: LEADER
I20260808 23:27:15.086773 15393 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9954b7c52019417da7fad83c7b23e589 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9954b7c52019417da7fad83c7b23e589" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 36003 } }
I20260808 23:27:15.087464 15386 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9954b7c52019417da7fad83c7b23e589 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:27:15.089442 15394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9954b7c52019417da7fad83c7b23e589 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9954b7c52019417da7fad83c7b23e589" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9954b7c52019417da7fad83c7b23e589" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 36003 } } }
I20260808 23:27:15.089825 15394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9954b7c52019417da7fad83c7b23e589 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:15.090503 15395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9954b7c52019417da7fad83c7b23e589 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9954b7c52019417da7fad83c7b23e589. Latest consensus state: current_term: 1 leader_uuid: "9954b7c52019417da7fad83c7b23e589" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9954b7c52019417da7fad83c7b23e589" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 36003 } } }
I20260808 23:27:15.091322 15395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9954b7c52019417da7fad83c7b23e589 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:15.091398 15400 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:27:15.095397 15400 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:27:15.104393 15400 catalog_manager.cc:1347] Generated new cluster ID: 4da4c642f8f5413c9ea5c96cc9860faf
I20260808 23:27:15.104491 15400 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:27:15.145442 15400 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:27:15.147066 15400 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:27:15.159284 15400 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9954b7c52019417da7fad83c7b23e589: Generated new TSK 0
I20260808 23:27:15.160089 15400 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.44.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
I20260808 23:27:15.206233 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:0
--local_ip_for_outbound_sockets=127.9.44.65
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:36003
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:15.520581 15421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:15.520821 15421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:15.520949 15421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:15.527808 15421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:15.527909 15421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:15.528060 15421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:27:15.537212 15421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:36003
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:15.538308 15421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:15.539484 15421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:15.551259 15428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:15.551224 15429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:15.551259 15431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:15.552414 15421 server_base.cc:1034] running on GCE node
I20260808 23:27:15.552956 15421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:15.553776 15421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:15.554966 15421 hybrid_clock.cc:648] HybridClock initialized: now 1786231635554919 us; error 41 us; skew 500 ppm
I20260808 23:27:15.555248 15421 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:15 dist-test-slave-6bbx krb5kdc[15269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231635, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:15.558148 15421 init.cc:377] Logged in from keytab as kudu/127.9.44.65@KRBTEST.COM (short username kudu)
I20260808 23:27:15.559156 15421 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:15.559255 15421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:15.559542 15421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:15.562240 15421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-0/data/instance:
uuid: "62b90ab846a54095afbae94e996f18a3"
format_stamp: "Formatted at 2026-08-08 23:27:15 on dist-test-slave-6bbx"
server_key: "4a09bca227564ccf68e6fdc9532dd54b"
server_key_iv: "0b033b4b64a06bc54f578bc396f20703"
server_key_version: "encryptionkey@0"
I20260808 23:27:15.562992 15421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance:
uuid: "62b90ab846a54095afbae94e996f18a3"
format_stamp: "Formatted at 2026-08-08 23:27:15 on dist-test-slave-6bbx"
server_key: "4a09bca227564ccf68e6fdc9532dd54b"
server_key_iv: "0b033b4b64a06bc54f578bc396f20703"
server_key_version: "encryptionkey@0"
I20260808 23:27:15.567545 15421 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260808 23:27:15.570773 15440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:15.572029 15421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:27:15.572175 15421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "62b90ab846a54095afbae94e996f18a3"
format_stamp: "Formatted at 2026-08-08 23:27:15 on dist-test-slave-6bbx"
server_key: "4a09bca227564ccf68e6fdc9532dd54b"
server_key_iv: "0b033b4b64a06bc54f578bc396f20703"
server_key_version: "encryptionkey@0"
I20260808 23:27:15.572371 15421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:15.593290 15421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:15.596834 15421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:15.597059 15421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:15.597908 15421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:15.599104 15421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:15.599200 15421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:15.599304 15421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:15.599375 15421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:15.628918 15421 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:41351
I20260808 23:27:15.628983 15621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:41351 every 8 connection(s)
I20260808 23:27:15.629642 15421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:27:15.633289 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 15421
I20260808 23:27:15.633508 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance
I20260808 23:27:15.634102 9393 external_mini_cluster.cc:1428] Setting key 602396880d7c66e542ccd7e37907ff61
Aug 08 23:27:15 dist-test-slave-6bbx krb5kdc[15269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231635, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:15.648350 15625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:15.631824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:46565 (local address 127.9.44.126:36003)
0808 23:27:15.632239 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:15.632243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:15.633146 (+ 903us) server_negotiation.cc:407] Connection header received
0808 23:27:15.635037 (+ 1891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:15.635042 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:15.635126 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:15.635249 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:15.636751 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:15.637493 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:15.638997 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:15.639386 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:15.639495 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:15.642701 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:15.642706 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:15.642709 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:15.642740 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:15.645374 (+ 2634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:15.646180 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:15.646184 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:15.646186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:15.646326 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:15.646739 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:15.646742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:15.646744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:15.646992 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:15.647097 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:15.647923 (+ 826us) server_negotiation.cc:299] Negotiation successful
0808 23:27:15.648090 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":210,"threads_started":1}
I20260808 23:27:15.649545 15626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:15.632299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:36003 (local address 127.9.44.65:46565)
0808 23:27:15.633001 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:15.633038 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:15.634733 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:15.635433 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:15.635441 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:15.636061 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:15.636586 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:15.636603 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:15.637644 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:15.637648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:15.638852 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:15.638865 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:15.639597 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:15.639605 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:15.639838 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:15.640421 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:15.640441 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:15.642488 (+ 2047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:15.645506 (+ 3018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:15.645515 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:15.645528 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:15.646038 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:15.646424 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:15.646429 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:15.646432 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:15.646646 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:15.647129 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:15.647137 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:15.647522 (+ 385us) client_negotiation.cc:769] Sending connection context
0808 23:27:15.647841 (+ 319us) client_negotiation.cc:240] Negotiation successful
0808 23:27:15.648306 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":278,"threads_started":1}
I20260808 23:27:15.650918 15623 heartbeater.cc:347] Connected to a master server at 127.9.44.126:36003
I20260808 23:27:15.651221 15623 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:15.651921 15623 heartbeater.cc:511] Master 127.9.44.126:36003 requested a full tablet report, sending...
I20260808 23:27:15.654033 15334 ts_manager.cc:194] Registered new tserver with Master: 62b90ab846a54095afbae94e996f18a3 (127.9.44.65:41351)
I20260808 23:27:15.655416 15334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.65@KRBTEST.COM'} at 127.9.44.65:46565
WARNING: no policy specified for kudu/127.9.44.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
I20260808 23:27:15.681869 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:0
--local_ip_for_outbound_sockets=127.9.44.66
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:36003
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:15.997025 15629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:15.997316 15629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:15.997426 15629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:16.004303 15629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:16.004396 15629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:16.004509 15629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:27:16.014770 15629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:36003
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:16.016041 15629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:16.017273 15629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:16.029105 15641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:16.029126 15640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:16.029125 15643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:16.029907 15629 server_base.cc:1034] running on GCE node
I20260808 23:27:16.030441 15629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:16.031271 15629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:16.032472 15629 hybrid_clock.cc:648] HybridClock initialized: now 1786231636032421 us; error 36 us; skew 500 ppm
I20260808 23:27:16.032758 15629 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:16 dist-test-slave-6bbx krb5kdc[15269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231636, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:16.035683 15629 init.cc:377] Logged in from keytab as kudu/127.9.44.66@KRBTEST.COM (short username kudu)
I20260808 23:27:16.036736 15629 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:16.036836 15629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:16.037133 15629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:16.039925 15629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-1/data/instance:
uuid: "4f55c37d31b1427088ac0bb30f740578"
format_stamp: "Formatted at 2026-08-08 23:27:16 on dist-test-slave-6bbx"
server_key: "a80ebeeb0fd8e20d3d1bdc5aca8bacaa"
server_key_iv: "b758eb49b7f0de16e06094c8ec1e1153"
server_key_version: "encryptionkey@0"
I20260808 23:27:16.040684 15629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance:
uuid: "4f55c37d31b1427088ac0bb30f740578"
format_stamp: "Formatted at 2026-08-08 23:27:16 on dist-test-slave-6bbx"
server_key: "a80ebeeb0fd8e20d3d1bdc5aca8bacaa"
server_key_iv: "b758eb49b7f0de16e06094c8ec1e1153"
server_key_version: "encryptionkey@0"
I20260808 23:27:16.045403 15629 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 23:27:16.048764 15652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:16.050024 15629 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 23:27:16.050166 15629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "4f55c37d31b1427088ac0bb30f740578"
format_stamp: "Formatted at 2026-08-08 23:27:16 on dist-test-slave-6bbx"
server_key: "a80ebeeb0fd8e20d3d1bdc5aca8bacaa"
server_key_iv: "b758eb49b7f0de16e06094c8ec1e1153"
server_key_version: "encryptionkey@0"
I20260808 23:27:16.050366 15629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:16.064297 15629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:16.067767 15629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:16.068002 15629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:16.068859 15629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:16.070081 15629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:16.070175 15629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:16.070289 15629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:16.070358 15629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:16.099545 15629 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:33317
I20260808 23:27:16.099602 15826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:33317 every 8 connection(s)
I20260808 23:27:16.100453 15629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:27:16.107406 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 15629
I20260808 23:27:16.107575 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance
I20260808 23:27:16.108060 9393 external_mini_cluster.cc:1428] Setting key 822494c125f2c8271731f670e0a18680
Aug 08 23:27:16 dist-test-slave-6bbx krb5kdc[15269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231636, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:16.118984 15625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:16.102756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:50893 (local address 127.9.44.126:36003)
0808 23:27:16.102962 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:16.102968 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:16.104055 (+ 1087us) server_negotiation.cc:407] Connection header received
0808 23:27:16.105739 (+ 1684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:16.105744 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:16.105814 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:16.105897 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:16.107364 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.108279 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.109991 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.110420 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.110531 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:16.113772 (+ 3241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:16.113777 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:16.113780 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:16.113811 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:16.116179 (+ 2368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:16.116845 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:16.116850 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:16.116853 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:16.116981 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:16.117385 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:16.117388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:16.117390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:16.117649 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:16.117783 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:16.118427 (+ 644us) server_negotiation.cc:299] Negotiation successful
0808 23:27:16.118674 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 23:27:16.119920 15831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:16.103187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:36003 (local address 127.9.44.66:50893)
0808 23:27:16.103916 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:16.103953 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:16.105489 (+ 1536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:16.106090 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:16.106097 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:16.106601 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:16.107193 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:16.107216 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.108429 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.108434 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:16.109850 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:16.109871 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.110639 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.110646 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:16.110872 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:16.111486 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:16.111510 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:16.113565 (+ 2055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:16.116312 (+ 2747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:16.116321 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:16.116335 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:16.116727 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:16.117087 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:16.117092 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:16.117095 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:16.117266 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:16.117773 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:16.117781 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:16.118079 (+ 298us) client_negotiation.cc:769] Sending connection context
0808 23:27:16.118310 (+ 231us) client_negotiation.cc:240] Negotiation successful
0808 23:27:16.118756 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":270,"threads_started":1}
I20260808 23:27:16.121302 15827 heartbeater.cc:347] Connected to a master server at 127.9.44.126:36003
I20260808 23:27:16.121562 15827 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:16.122156 15827 heartbeater.cc:511] Master 127.9.44.126:36003 requested a full tablet report, sending...
I20260808 23:27:16.123569 15334 ts_manager.cc:194] Registered new tserver with Master: 4f55c37d31b1427088ac0bb30f740578 (127.9.44.66:33317)
I20260808 23:27:16.124639 15334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.66@KRBTEST.COM'} at 127.9.44.66:50893
WARNING: no policy specified for kudu/127.9.44.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
I20260808 23:27:16.151969 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:0
--local_ip_for_outbound_sockets=127.9.44.67
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:36003
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:16.466383 15836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:16.466634 15836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:16.466737 15836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:16.473611 15836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:16.473703 15836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:16.473816 15836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:27:16.483024 15836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:36003
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:16.484149 15836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:16.485345 15836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:16.497336 15847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:16.497284 15846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:16.497510 15849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:16.498406 15836 server_base.cc:1034] running on GCE node
I20260808 23:27:16.498962 15836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:16.499845 15836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:16.501042 15836 hybrid_clock.cc:648] HybridClock initialized: now 1786231636500996 us; error 35 us; skew 500 ppm
I20260808 23:27:16.501333 15836 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:16 dist-test-slave-6bbx krb5kdc[15269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231636, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:16.504225 15836 init.cc:377] Logged in from keytab as kudu/127.9.44.67@KRBTEST.COM (short username kudu)
I20260808 23:27:16.505249 15836 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:16.505350 15836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:16.505653 15836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:16.508499 15836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-2/data/instance:
uuid: "b50b10d25edf48cdad343968fbc85165"
format_stamp: "Formatted at 2026-08-08 23:27:16 on dist-test-slave-6bbx"
server_key: "951cb0b96af19c52201ea3ffcebf33f9"
server_key_iv: "311acd2d75013afe66f927ab0a2179c3"
server_key_version: "encryptionkey@0"
I20260808 23:27:16.509262 15836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance:
uuid: "b50b10d25edf48cdad343968fbc85165"
format_stamp: "Formatted at 2026-08-08 23:27:16 on dist-test-slave-6bbx"
server_key: "951cb0b96af19c52201ea3ffcebf33f9"
server_key_iv: "311acd2d75013afe66f927ab0a2179c3"
server_key_version: "encryptionkey@0"
I20260808 23:27:16.513907 15836 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260808 23:27:16.517187 15857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:16.518323 15836 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 23:27:16.518496 15836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "b50b10d25edf48cdad343968fbc85165"
format_stamp: "Formatted at 2026-08-08 23:27:16 on dist-test-slave-6bbx"
server_key: "951cb0b96af19c52201ea3ffcebf33f9"
server_key_iv: "311acd2d75013afe66f927ab0a2179c3"
server_key_version: "encryptionkey@0"
I20260808 23:27:16.518688 15836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:16.540596 15836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:16.544018 15836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:16.544242 15836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:16.545099 15836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:16.546308 15836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:16.546413 15836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:16.546517 15836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:16.546587 15836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:16.576018 15836 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:38523
I20260808 23:27:16.576078 16028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:38523 every 8 connection(s)
I20260808 23:27:16.576963 15836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:27:16.577571 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 15836
I20260808 23:27:16.577760 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance
I20260808 23:27:16.578289 9393 external_mini_cluster.cc:1428] Setting key bf369a9340dbb6780a3489d5e49519d3
Aug 08 23:27:16 dist-test-slave-6bbx krb5kdc[15269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231636, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:16.595448 15625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:16.579469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:39525 (local address 127.9.44.126:36003)
0808 23:27:16.579795 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:16.579801 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:16.580994 (+ 1193us) server_negotiation.cc:407] Connection header received
0808 23:27:16.582810 (+ 1816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:16.582814 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:16.582889 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:16.582967 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:16.584332 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.585075 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.586479 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.586848 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.586940 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:16.589944 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:16.589949 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:16.589952 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:16.589983 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:16.592461 (+ 2478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:16.593277 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:16.593283 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:16.593286 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:16.593408 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:16.593819 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:16.593824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:16.593826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:16.594118 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:16.594255 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:16.594906 (+ 651us) server_negotiation.cc:299] Negotiation successful
0808 23:27:16.595114 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200}
I20260808 23:27:16.596352 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:16.580017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:36003 (local address 127.9.44.67:39525)
0808 23:27:16.580834 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:16.580886 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:16.582570 (+ 1684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:16.583135 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:16.583142 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:16.583615 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:16.584190 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:16.584208 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.585190 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.585193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:16.586376 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:16.586389 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.587008 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.587013 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:16.587180 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:16.587786 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:16.587807 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:16.589761 (+ 1954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:16.592581 (+ 2820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:16.592590 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:16.592603 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:16.593153 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:16.593519 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:16.593524 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:16.593528 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:16.593699 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:16.594251 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:16.594258 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:16.594550 (+ 292us) client_negotiation.cc:769] Sending connection context
0808 23:27:16.594771 (+ 221us) client_negotiation.cc:240] Negotiation successful
0808 23:27:16.595242 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":363,"threads_started":1}
I20260808 23:27:16.597564 16029 heartbeater.cc:347] Connected to a master server at 127.9.44.126:36003
I20260808 23:27:16.597811 16029 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:16.598366 16029 heartbeater.cc:511] Master 127.9.44.126:36003 requested a full tablet report, sending...
I20260808 23:27:16.599635 15334 ts_manager.cc:194] Registered new tserver with Master: b50b10d25edf48cdad343968fbc85165 (127.9.44.67:38523)
I20260808 23:27:16.600435 15334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.67@KRBTEST.COM'} at 127.9.44.67:39525
I20260808 23:27:16.605310 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:27:16.620282 15625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:16.609560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52546 (local address 127.9.44.126:36003)
0808 23:27:16.609716 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:16.609722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:16.610234 (+ 512us) server_negotiation.cc:407] Connection header received
0808 23:27:16.610362 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:16.610366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:16.610421 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:16.610493 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:16.611755 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.612410 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.613553 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.613971 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.614056 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:16.616052 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:16.616056 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:16.616059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:16.616092 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:16.618189 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:16.618728 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:16.618731 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:16.618733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:16.618869 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:16.619109 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:16.619112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:16.619114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:16.619377 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:16.619543 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:16.619832 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 23:27:16.619989 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 23:27:16.623716 15334 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52546:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:27:16.630175 15334 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:27:16.652738 16049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:16.641290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:41351 (local address 127.0.0.1:41054)
0808 23:27:16.643307 (+ 2017us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:16.643321 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:16.643470 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:16.644354 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:16.644373 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:16.644400 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:16.644841 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:16.644852 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.646926 (+ 2074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.646933 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:16.649400 (+ 2467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:16.649418 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.651634 (+ 2216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.651641 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:16.651880 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:16.651937 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 23:27:16.652066 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 23:27:16.652246 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1917,"mutex_wait_us":1,"thread_start_us":1761,"threads_started":1}
I20260808 23:27:16.653702 16052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:16.641794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41054 (local address 127.9.44.65:41351)
0808 23:27:16.642895 (+ 1101us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:16.642903 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:16.643606 (+ 703us) server_negotiation.cc:407] Connection header received
0808 23:27:16.643727 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:16.643734 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:16.643938 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:16.644119 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:16.645053 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.646715 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.649603 (+ 2888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.651116 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.651244 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:16.651496 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:16.653095 (+ 1599us) server_negotiation.cc:299] Negotiation successful
0808 23:27:16.653278 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":927,"thread_start_us":784,"threads_started":1}
I20260808 23:27:16.653430 16048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:16.640922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:33317 (local address 127.0.0.1:59126)
0808 23:27:16.641825 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:16.641867 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:16.642045 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:16.643019 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:16.643027 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:16.643061 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:16.643529 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:16.643545 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.646664 (+ 3119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.646670 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:16.649389 (+ 2719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:16.649407 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.652532 (+ 3125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.652539 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:16.652756 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:16.652792 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 23:27:16.652879 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 23:27:16.653009 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"mutex_wait_us":72,"thread_start_us":540,"threads_started":1}
I20260808 23:27:16.655233 16050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:16.641546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59126 (local address 127.9.44.66:33317)
0808 23:27:16.642383 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:16.642392 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:16.642419 (+ 27us) server_negotiation.cc:407] Connection header received
0808 23:27:16.642512 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:16.642521 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:16.642756 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:16.642970 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:16.643760 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.646492 (+ 2732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.650549 (+ 4057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.652064 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.652187 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:16.652366 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:16.653562 (+ 1196us) server_negotiation.cc:299] Negotiation successful
0808 23:27:16.654887 (+ 1325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":689,"thread_start_us":720,"threads_started":1}
I20260808 23:27:16.658614 15623 heartbeater.cc:503] Master 127.9.44.126:36003 was elected leader, sending a full tablet report...
I20260808 23:27:16.658879 15730 tablet_service.cc:1467] Processing CreateTablet for tablet bd73948a968f49a9af36787798dc31fd (DEFAULT_TABLE table=test-table [id=7bfcb5c473b0400fad4240e8c59456e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:27:16.659705 15527 tablet_service.cc:1467] Processing CreateTablet for tablet bd73948a968f49a9af36787798dc31fd (DEFAULT_TABLE table=test-table [id=7bfcb5c473b0400fad4240e8c59456e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:27:16.660354 15730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd73948a968f49a9af36787798dc31fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:16.661072 15527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd73948a968f49a9af36787798dc31fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:16.661305 16048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:16.643443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:38523 (local address 127.0.0.1:51564)
0808 23:27:16.654413 (+ 10970us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:16.654430 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:16.654515 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:16.655651 (+ 1136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:16.655655 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:16.655705 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:16.655867 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:16.655879 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.657592 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.657596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:16.658820 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:16.658835 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.660579 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.660585 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:16.660758 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:16.660789 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 23:27:16.660892 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 23:27:16.661026 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10873,"thread_start_us":12679,"threads_started":1}
I20260808 23:27:16.661504 16054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:16.643827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51564 (local address 127.9.44.67:38523)
0808 23:27:16.655176 (+ 11349us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:16.655181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:16.655205 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:27:16.655272 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:16.655276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:16.655483 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:16.655645 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:16.656085 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.657457 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.659002 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.660384 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.660549 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:16.660765 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:16.661027 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 23:27:16.661187 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11218,"thread_start_us":11051,"threads_started":1}
I20260808 23:27:16.664322 15933 tablet_service.cc:1467] Processing CreateTablet for tablet bd73948a968f49a9af36787798dc31fd (DEFAULT_TABLE table=test-table [id=7bfcb5c473b0400fad4240e8c59456e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:27:16.665341 15933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd73948a968f49a9af36787798dc31fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:16.675433 16057 tablet_bootstrap.cc:492] T bd73948a968f49a9af36787798dc31fd P 62b90ab846a54095afbae94e996f18a3: Bootstrap starting.
I20260808 23:27:16.677507 16059 tablet_bootstrap.cc:492] T bd73948a968f49a9af36787798dc31fd P b50b10d25edf48cdad343968fbc85165: Bootstrap starting.
I20260808 23:27:16.678584 16056 tablet_bootstrap.cc:492] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578: Bootstrap starting.
I20260808 23:27:16.679354 16057 tablet_bootstrap.cc:654] T bd73948a968f49a9af36787798dc31fd P 62b90ab846a54095afbae94e996f18a3: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:16.680728 16059 tablet_bootstrap.cc:654] T bd73948a968f49a9af36787798dc31fd P b50b10d25edf48cdad343968fbc85165: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:16.680687 16057 log.cc:824] T bd73948a968f49a9af36787798dc31fd P 62b90ab846a54095afbae94e996f18a3: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:16.681818 16056 tablet_bootstrap.cc:654] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:16.681800 16059 log.cc:824] T bd73948a968f49a9af36787798dc31fd P b50b10d25edf48cdad343968fbc85165: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:16.683000 16056 log.cc:824] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:16.683943 16057 tablet_bootstrap.cc:492] T bd73948a968f49a9af36787798dc31fd P 62b90ab846a54095afbae94e996f18a3: No bootstrap required, opened a new log
I20260808 23:27:16.684249 16057 ts_tablet_manager.cc:1397] T bd73948a968f49a9af36787798dc31fd P 62b90ab846a54095afbae94e996f18a3: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260808 23:27:16.685792 16056 tablet_bootstrap.cc:492] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578: No bootstrap required, opened a new log
I20260808 23:27:16.686051 16056 ts_tablet_manager.cc:1397] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260808 23:27:16.686298 16059 tablet_bootstrap.cc:492] T bd73948a968f49a9af36787798dc31fd P b50b10d25edf48cdad343968fbc85165: No bootstrap required, opened a new log
I20260808 23:27:16.686563 16059 ts_tablet_manager.cc:1397] T bd73948a968f49a9af36787798dc31fd P b50b10d25edf48cdad343968fbc85165: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260808 23:27:16.691211 16057 raft_consensus.cc:348] T bd73948a968f49a9af36787798dc31fd P 62b90ab846a54095afbae94e996f18a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f55c37d31b1427088ac0bb30f740578" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33317 } } peers { permanent_uuid: "62b90ab846a54095afbae94e996f18a3" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 41351 } } peers { permanent_uuid: "b50b10d25edf48cdad343968fbc85165" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38523 } }
I20260808 23:27:16.691725 16057 raft_consensus.cc:374] T bd73948a968f49a9af36787798dc31fd P 62b90ab846a54095afbae94e996f18a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:16.691892 16057 raft_consensus.cc:729] T bd73948a968f49a9af36787798dc31fd P 62b90ab846a54095afbae94e996f18a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62b90ab846a54095afbae94e996f18a3, State: Initialized, Role: FOLLOWER
I20260808 23:27:16.692610 16057 consensus_queue.cc:260] T bd73948a968f49a9af36787798dc31fd P 62b90ab846a54095afbae94e996f18a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f55c37d31b1427088ac0bb30f740578" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33317 } } peers { permanent_uuid: "62b90ab846a54095afbae94e996f18a3" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 41351 } } peers { permanent_uuid: "b50b10d25edf48cdad343968fbc85165" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38523 } }
I20260808 23:27:16.696069 16057 ts_tablet_manager.cc:1428] T bd73948a968f49a9af36787798dc31fd P 62b90ab846a54095afbae94e996f18a3: Time spent starting tablet: real 0.012s user 0.010s sys 0.001s
I20260808 23:27:16.695977 16056 raft_consensus.cc:348] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f55c37d31b1427088ac0bb30f740578" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33317 } } peers { permanent_uuid: "62b90ab846a54095afbae94e996f18a3" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 41351 } } peers { permanent_uuid: "b50b10d25edf48cdad343968fbc85165" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38523 } }
I20260808 23:27:16.696472 16056 raft_consensus.cc:374] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:16.696584 16056 raft_consensus.cc:729] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f55c37d31b1427088ac0bb30f740578, State: Initialized, Role: FOLLOWER
I20260808 23:27:16.696537 16059 raft_consensus.cc:348] T bd73948a968f49a9af36787798dc31fd P b50b10d25edf48cdad343968fbc85165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f55c37d31b1427088ac0bb30f740578" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33317 } } peers { permanent_uuid: "62b90ab846a54095afbae94e996f18a3" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 41351 } } peers { permanent_uuid: "b50b10d25edf48cdad343968fbc85165" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38523 } }
I20260808 23:27:16.696920 16059 raft_consensus.cc:374] T bd73948a968f49a9af36787798dc31fd P b50b10d25edf48cdad343968fbc85165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:16.697007 16059 raft_consensus.cc:729] T bd73948a968f49a9af36787798dc31fd P b50b10d25edf48cdad343968fbc85165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b50b10d25edf48cdad343968fbc85165, State: Initialized, Role: FOLLOWER
I20260808 23:27:16.697163 16056 consensus_queue.cc:260] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f55c37d31b1427088ac0bb30f740578" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33317 } } peers { permanent_uuid: "62b90ab846a54095afbae94e996f18a3" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 41351 } } peers { permanent_uuid: "b50b10d25edf48cdad343968fbc85165" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38523 } }
I20260808 23:27:16.697548 16059 consensus_queue.cc:260] T bd73948a968f49a9af36787798dc31fd P b50b10d25edf48cdad343968fbc85165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f55c37d31b1427088ac0bb30f740578" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33317 } } peers { permanent_uuid: "62b90ab846a54095afbae94e996f18a3" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 41351 } } peers { permanent_uuid: "b50b10d25edf48cdad343968fbc85165" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38523 } }
I20260808 23:27:16.698249 15827 heartbeater.cc:503] Master 127.9.44.126:36003 was elected leader, sending a full tablet report...
I20260808 23:27:16.698853 16056 ts_tablet_manager.cc:1428] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578: Time spent starting tablet: real 0.013s user 0.010s sys 0.003s
I20260808 23:27:16.699878 16029 heartbeater.cc:503] Master 127.9.44.126:36003 was elected leader, sending a full tablet report...
I20260808 23:27:16.700314 16059 ts_tablet_manager.cc:1428] T bd73948a968f49a9af36787798dc31fd P b50b10d25edf48cdad343968fbc85165: Time spent starting tablet: real 0.014s user 0.006s sys 0.010s
W20260808 23:27:16.828540 16031 tablet.cc:2369] T bd73948a968f49a9af36787798dc31fd P b50b10d25edf48cdad343968fbc85165: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:27:16.852280 15828 tablet.cc:2369] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:27:16.878849 16064 raft_consensus.cc:482] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:27:16.879137 16064 raft_consensus.cc:504] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f55c37d31b1427088ac0bb30f740578" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33317 } } peers { permanent_uuid: "62b90ab846a54095afbae94e996f18a3" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 41351 } } peers { permanent_uuid: "b50b10d25edf48cdad343968fbc85165" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38523 } }
I20260808 23:27:16.880859 16064 leader_election.cc:288] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62b90ab846a54095afbae94e996f18a3 (127.9.44.65:41351), b50b10d25edf48cdad343968fbc85165 (127.9.44.67:38523)
W20260808 23:27:16.881614 15624 tablet.cc:2369] T bd73948a968f49a9af36787798dc31fd P 62b90ab846a54095afbae94e996f18a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:27:16.887266 16052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:16.881196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:60189 (local address 127.9.44.65:41351)
0808 23:27:16.881391 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:16.881398 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:16.881897 (+ 499us) server_negotiation.cc:407] Connection header received
0808 23:27:16.882226 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:16.882234 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:16.882323 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:16.882389 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:16.883151 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.883918 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.885629 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.886377 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.886498 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:16.886597 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:16.886893 (+ 296us) server_negotiation.cc:299] Negotiation successful
0808 23:27:16.887014 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 23:27:16.887333 16066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:16.881141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:41351 (local address 127.9.44.66:60189)
0808 23:27:16.881800 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:16.881818 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:16.882018 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:16.882612 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:16.882616 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:16.882658 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:16.883001 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:16.883011 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.884041 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.884046 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:16.885475 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:16.885488 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.886593 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.886599 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:16.886743 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:16.886766 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 23:27:16.886848 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 23:27:16.886967 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":297,"threads_started":1}
I20260808 23:27:16.887900 16054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:16.882171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:52315 (local address 127.9.44.67:38523)
0808 23:27:16.882382 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:16.882388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:16.882933 (+ 545us) server_negotiation.cc:407] Connection header received
0808 23:27:16.883099 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:16.883102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:16.883159 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:16.883246 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:16.883783 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.884491 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.886144 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.886927 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.887039 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:16.887134 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:16.887533 (+ 399us) server_negotiation.cc:299] Negotiation successful
0808 23:27:16.887657 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 23:27:16.888563 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:16.881890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:38523 (local address 127.9.44.66:52315)
0808 23:27:16.882812 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:16.882833 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:16.882973 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:16.883271 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:16.883276 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:16.883299 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:16.883613 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:16.883621 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.884625 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.884629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:16.885988 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:16.886001 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.887149 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.887156 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:16.887293 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:16.887317 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 23:27:16.888125 (+ 808us) client_negotiation.cc:240] Negotiation successful
0808 23:27:16.888252 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":813,"thread_start_us":639,"threads_started":1}
I20260808 23:27:16.888384 15552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd73948a968f49a9af36787798dc31fd" candidate_uuid: "4f55c37d31b1427088ac0bb30f740578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62b90ab846a54095afbae94e996f18a3" is_pre_election: true
I20260808 23:27:16.888942 15552 raft_consensus.cc:2454] T bd73948a968f49a9af36787798dc31fd P 62b90ab846a54095afbae94e996f18a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f55c37d31b1427088ac0bb30f740578 in term 0.
I20260808 23:27:16.889488 15961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd73948a968f49a9af36787798dc31fd" candidate_uuid: "4f55c37d31b1427088ac0bb30f740578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50b10d25edf48cdad343968fbc85165" is_pre_election: true
I20260808 23:27:16.889587 15657 leader_election.cc:302] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f55c37d31b1427088ac0bb30f740578, 62b90ab846a54095afbae94e996f18a3; no voters:
I20260808 23:27:16.889981 16064 raft_consensus.cc:2781] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:27:16.889979 15961 raft_consensus.cc:2454] T bd73948a968f49a9af36787798dc31fd P b50b10d25edf48cdad343968fbc85165 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f55c37d31b1427088ac0bb30f740578 in term 0.
I20260808 23:27:16.890141 16064 raft_consensus.cc:482] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:27:16.890247 16064 raft_consensus.cc:3037] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:16.892465 16064 raft_consensus.cc:504] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f55c37d31b1427088ac0bb30f740578" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33317 } } peers { permanent_uuid: "62b90ab846a54095afbae94e996f18a3" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 41351 } } peers { permanent_uuid: "b50b10d25edf48cdad343968fbc85165" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38523 } }
I20260808 23:27:16.892982 16064 leader_election.cc:288] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578 [CANDIDATE]: Term 1 election: Requested vote from peers 62b90ab846a54095afbae94e996f18a3 (127.9.44.65:41351), b50b10d25edf48cdad343968fbc85165 (127.9.44.67:38523)
I20260808 23:27:16.893402 15552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd73948a968f49a9af36787798dc31fd" candidate_uuid: "4f55c37d31b1427088ac0bb30f740578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62b90ab846a54095afbae94e996f18a3"
I20260808 23:27:16.893486 15961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd73948a968f49a9af36787798dc31fd" candidate_uuid: "4f55c37d31b1427088ac0bb30f740578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50b10d25edf48cdad343968fbc85165"
I20260808 23:27:16.893615 15552 raft_consensus.cc:3037] T bd73948a968f49a9af36787798dc31fd P 62b90ab846a54095afbae94e996f18a3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:16.893656 15961 raft_consensus.cc:3037] T bd73948a968f49a9af36787798dc31fd P b50b10d25edf48cdad343968fbc85165 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:16.896715 15552 raft_consensus.cc:2454] T bd73948a968f49a9af36787798dc31fd P 62b90ab846a54095afbae94e996f18a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f55c37d31b1427088ac0bb30f740578 in term 1.
I20260808 23:27:16.896715 15961 raft_consensus.cc:2454] T bd73948a968f49a9af36787798dc31fd P b50b10d25edf48cdad343968fbc85165 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f55c37d31b1427088ac0bb30f740578 in term 1.
I20260808 23:27:16.897110 15657 leader_election.cc:302] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f55c37d31b1427088ac0bb30f740578, b50b10d25edf48cdad343968fbc85165; no voters:
I20260808 23:27:16.897346 16064 raft_consensus.cc:2781] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:27:16.897764 16064 raft_consensus.cc:686] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578 [term 1 LEADER]: Becoming Leader. State: Replica: 4f55c37d31b1427088ac0bb30f740578, State: Running, Role: LEADER
I20260808 23:27:16.898214 16064 consensus_queue.cc:237] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f55c37d31b1427088ac0bb30f740578" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33317 } } peers { permanent_uuid: "62b90ab846a54095afbae94e996f18a3" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 41351 } } peers { permanent_uuid: "b50b10d25edf48cdad343968fbc85165" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38523 } }
I20260808 23:27:16.902778 15333 catalog_manager.cc:5495] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f55c37d31b1427088ac0bb30f740578 (127.9.44.66). New cstate: current_term: 1 leader_uuid: "4f55c37d31b1427088ac0bb30f740578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f55c37d31b1427088ac0bb30f740578" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62b90ab846a54095afbae94e996f18a3" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 41351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b50b10d25edf48cdad343968fbc85165" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38523 } health_report { overall_health: UNKNOWN } } }
I20260808 23:27:16.944639 16050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:16.939956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59130 (local address 127.9.44.66:33317)
0808 23:27:16.940150 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:16.940155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:16.940177 (+ 22us) server_negotiation.cc:407] Connection header received
0808 23:27:16.940370 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:16.940374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:16.940448 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:16.940583 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:27:16.940876 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.941573 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.942774 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:16.943151 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:16.943237 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:16.943608 (+ 371us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:27:16.944128 (+ 520us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:27:16.944234 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:16.944308 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 23:27:16.944380 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 23:27:16.952306 15961 raft_consensus.cc:1261] T bd73948a968f49a9af36787798dc31fd P b50b10d25edf48cdad343968fbc85165 [term 1 FOLLOWER]: Refusing update from remote peer 4f55c37d31b1427088ac0bb30f740578: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:27:16.952406 15552 raft_consensus.cc:1261] T bd73948a968f49a9af36787798dc31fd P 62b90ab846a54095afbae94e996f18a3 [term 1 FOLLOWER]: Refusing update from remote peer 4f55c37d31b1427088ac0bb30f740578: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:27:16.953228 16068 consensus_queue.cc:1035] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b50b10d25edf48cdad343968fbc85165" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:27:16.953601 16064 consensus_queue.cc:1035] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62b90ab846a54095afbae94e996f18a3" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 41351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:27:16.982030 15333 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:27:16.982331 15333 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:27:16.985750 15333 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9954b7c52019417da7fad83c7b23e589: Sending DeleteTablet for 3 replicas of tablet bd73948a968f49a9af36787798dc31fd
I20260808 23:27:16.986816 15730 tablet_service.cc:1514] Processing DeleteTablet for tablet bd73948a968f49a9af36787798dc31fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:27:16 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:59126
I20260808 23:27:16.986877 15527 tablet_service.cc:1514] Processing DeleteTablet for tablet bd73948a968f49a9af36787798dc31fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:27:16 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:41054
I20260808 23:27:16.987123 15933 tablet_service.cc:1514] Processing DeleteTablet for tablet bd73948a968f49a9af36787798dc31fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:27:16 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:51564
I20260808 23:27:16.988557 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 15421
I20260808 23:27:16.999140 16094 tablet_replica.cc:330] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578: stopping tablet replica
I20260808 23:27:16.999696 16094 raft_consensus.cc:2229] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:27:17.000412 16094 raft_consensus.cc:2258] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:27:17.001235 16097 tablet_replica.cc:330] T bd73948a968f49a9af36787798dc31fd P 62b90ab846a54095afbae94e996f18a3: stopping tablet replica
I20260808 23:27:17.001482 16098 tablet_replica.cc:330] T bd73948a968f49a9af36787798dc31fd P b50b10d25edf48cdad343968fbc85165: stopping tablet replica
I20260808 23:27:17.001852 16098 raft_consensus.cc:2229] T bd73948a968f49a9af36787798dc31fd P b50b10d25edf48cdad343968fbc85165 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:27:17.002341 16098 raft_consensus.cc:2258] T bd73948a968f49a9af36787798dc31fd P b50b10d25edf48cdad343968fbc85165 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:27:17.003250 16094 ts_tablet_manager.cc:1905] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:27:17.004365 16098 ts_tablet_manager.cc:1905] T bd73948a968f49a9af36787798dc31fd P b50b10d25edf48cdad343968fbc85165: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:27:17.004832 16097 raft_consensus.cc:2229] T bd73948a968f49a9af36787798dc31fd P 62b90ab846a54095afbae94e996f18a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:27:17.005342 16097 raft_consensus.cc:2258] T bd73948a968f49a9af36787798dc31fd P 62b90ab846a54095afbae94e996f18a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:27:17.008513 16097 ts_tablet_manager.cc:1905] T bd73948a968f49a9af36787798dc31fd P 62b90ab846a54095afbae94e996f18a3: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 23:27:17 dist-test-slave-6bbx krb5kdc[15269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.65@KRBTEST.COM
I20260808 23:27:17.011142 16098 ts_tablet_manager.cc:1918] T bd73948a968f49a9af36787798dc31fd P b50b10d25edf48cdad343968fbc85165: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:27:17.011328 16098 log.cc:1196] T bd73948a968f49a9af36787798dc31fd P b50b10d25edf48cdad343968fbc85165: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/bd73948a968f49a9af36787798dc31fd
I20260808 23:27:17.011893 16098 ts_tablet_manager.cc:1939] T bd73948a968f49a9af36787798dc31fd P b50b10d25edf48cdad343968fbc85165: Deleting consensus metadata
I20260808 23:27:17.014129 15310 catalog_manager.cc:4841] TS b50b10d25edf48cdad343968fbc85165 (127.9.44.67:38523): tablet bd73948a968f49a9af36787798dc31fd (table test-table [id=7bfcb5c473b0400fad4240e8c59456e7]) successfully deleted
I20260808 23:27:17.016232 16094 ts_tablet_manager.cc:1918] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:27:17.016417 16094 log.cc:1196] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/bd73948a968f49a9af36787798dc31fd
I20260808 23:27:17.017151 16094 ts_tablet_manager.cc:1939] T bd73948a968f49a9af36787798dc31fd P 4f55c37d31b1427088ac0bb30f740578: Deleting consensus metadata
I20260808 23:27:17.018689 15308 catalog_manager.cc:4841] TS 4f55c37d31b1427088ac0bb30f740578 (127.9.44.66:33317): tablet bd73948a968f49a9af36787798dc31fd (table test-table [id=7bfcb5c473b0400fad4240e8c59456e7]) successfully deleted
I20260808 23:27:17.019873 16052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:16.989707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41062 (local address 127.9.44.65:41351)
0808 23:27:16.989877 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:16.989883 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:17.000475 (+ 10592us) server_negotiation.cc:407] Connection header received
0808 23:27:17.000653 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:17.000659 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:17.000756 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:17.000866 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:17.002891 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:17.004069 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:17.005914 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:17.006543 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:17.006641 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:17.011882 (+ 5241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:17.011897 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:17.011914 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:17.011960 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:17.016191 (+ 4231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:17.017053 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:17.017063 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:17.017066 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:17.017237 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:17.017792 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:17.017798 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:17.017801 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:17.018280 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:17.018595 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:17.019192 (+ 597us) server_negotiation.cc:299] Negotiation successful
0808 23:27:17.019425 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 23:27:17.020684 15616 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:17.169191 16097 ts_tablet_manager.cc:1918] T bd73948a968f49a9af36787798dc31fd P 62b90ab846a54095afbae94e996f18a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:27:17.169474 16097 log.cc:1196] T bd73948a968f49a9af36787798dc31fd P 62b90ab846a54095afbae94e996f18a3: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786231597224934-9393-0/minicluster-data/ts-0/wal/wals/bd73948a968f49a9af36787798dc31fd
I20260808 23:27:17.169951 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 15421
I20260808 23:27:17.170053 16097 ts_tablet_manager.cc:1939] T bd73948a968f49a9af36787798dc31fd P 62b90ab846a54095afbae94e996f18a3: Deleting consensus metadata
W20260808 23:27:17.189872 15310 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.44.65:41351 (error 108)
I20260808 23:27:17.190987 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 15629
I20260808 23:27:17.191824 16102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:17.191048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:41351 (local address 127.0.0.1:41066)
0808 23:27:17.191483 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:17.191600 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:41351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":206,"threads_started":1}
W20260808 23:27:17.192202 15310 catalog_manager.cc:4584] TS 62b90ab846a54095afbae94e996f18a3 (127.9.44.65:41351): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bd73948a968f49a9af36787798dc31fd: Network error: Client connection negotiation failed: client connection to 127.9.44.65:41351: connect: Connection refused (error 111)
Aug 08 23:27:17 dist-test-slave-6bbx krb5kdc[15269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.66@KRBTEST.COM
I20260808 23:27:17.207935 16050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:17.191935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59138 (local address 127.9.44.66:33317)
0808 23:27:17.192658 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:17.192663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:17.192684 (+ 21us) server_negotiation.cc:407] Connection header received
0808 23:27:17.192746 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:17.192749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:17.192817 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:17.192894 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:17.194539 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:17.195343 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:17.197191 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:17.197689 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:17.197793 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:17.202391 (+ 4598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:17.202403 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:17.202417 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:17.202459 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:17.205255 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:17.205880 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:17.205887 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:17.205890 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:17.206036 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:17.206403 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:17.206407 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:17.206409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:17.206757 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:17.207098 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:17.207305 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 23:27:17.207510 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617}
I20260808 23:27:17.208653 15820 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:17.245831 16102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:17.245403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:41351 (local address 127.0.0.1:41070)
0808 23:27:17.245589 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:17.245683 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:41351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 23:27:17.246811 16102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:17.246427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:41351 (local address 127.0.0.1:41082)
0808 23:27:17.246592 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:17.246662 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:41351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 23:27:17.307041 16102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:17.306620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:41351 (local address 127.0.0.1:41098)
0808 23:27:17.306810 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:17.306893 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:41351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 23:27:17.308034 16102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:17.307686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:41351 (local address 127.0.0.1:41114)
0808 23:27:17.307814 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:17.307889 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:41351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 23:27:17.318127 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 15629
I20260808 23:27:17.343691 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 15836
Aug 08 23:27:17 dist-test-slave-6bbx krb5kdc[15269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.67@KRBTEST.COM
I20260808 23:27:17.357512 16054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:17.344500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51576 (local address 127.9.44.67:38523)
0808 23:27:17.344666 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:17.344671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:17.344703 (+ 32us) server_negotiation.cc:407] Connection header received
0808 23:27:17.344866 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:17.344870 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:17.344935 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:17.345018 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:17.346185 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:17.346889 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:17.348166 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:17.348562 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:17.348655 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:17.352509 (+ 3854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:17.352522 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:17.352539 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:17.352578 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:17.355234 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:17.355848 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:17.355855 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:17.355858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:17.355980 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:17.356279 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:17.356282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:17.356284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:17.356640 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:17.356859 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:17.357042 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 23:27:17.357199 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 23:27:17.358105 16023 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:17.388361 16102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:17.387863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:41351 (local address 127.0.0.1:41120)
0808 23:27:17.388090 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:17.388191 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:41351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260808 23:27:17.389379 16102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:17.388961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:41351 (local address 127.0.0.1:41134)
0808 23:27:17.389166 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:17.389236 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:41351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260808 23:27:17.444494 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 15836
I20260808 23:27:17.466940 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 15288
I20260808 23:27:17.468124 15379 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:17.572018 16102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:17.571433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:41351 (local address 127.0.0.1:41142)
0808 23:27:17.571693 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:17.571859 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:41351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260808 23:27:17.572306 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 15288
2026-08-08T23:27:17Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4285 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T23:27:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:27:17Z Disabled control of system clock
I20260808 23:27:17.645267 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:35533
--webserver_interface=127.9.44.126
--webserver_port=0
--builtin_ntp_servers=127.9.44.84:38701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:35533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 23:27:17.957293 16114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:17.957564 16114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:17.957695 16114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:17.964710 16114 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:27:17.964807 16114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:17.964871 16114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:17.964944 16114 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:27:17.965003 16114 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:27:17.973812 16114 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:38701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:35533
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:35533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:17.974946 16114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:17.976632 16114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:17.987658 16123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:17.987715 16124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:17.988112 16126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:17.988379 16114 server_base.cc:1034] running on GCE node
I20260808 23:27:17.989084 16114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:17.990304 16114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:17.991495 16114 hybrid_clock.cc:648] HybridClock initialized: now 1786231637991449 us; error 37 us; skew 500 ppm
I20260808 23:27:17.991827 16114 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:27:17.993888 16114 webserver.cc:460] Webserver started at http://127.9.44.126:39655/ using document root <none> and password file <none>
I20260808 23:27:17.994495 16114 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:17.994580 16114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:17.994840 16114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:17.997627 16114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "c28c75dc9d52407d84e4af4541a024b0"
format_stamp: "Formatted at 2026-08-08 23:27:17 on dist-test-slave-6bbx"
server_key: "62011c01b30a711c0a9a3f4abb76e314"
server_key_iv: "55dc07c104cf654ae19ec11afcbb614e"
server_key_version: "encryptionkey@0"
I20260808 23:27:17.998337 16114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "c28c75dc9d52407d84e4af4541a024b0"
format_stamp: "Formatted at 2026-08-08 23:27:17 on dist-test-slave-6bbx"
server_key: "62011c01b30a711c0a9a3f4abb76e314"
server_key_iv: "55dc07c104cf654ae19ec11afcbb614e"
server_key_version: "encryptionkey@0"
I20260808 23:27:18.003017 16114 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260808 23:27:18.006234 16137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:18.007445 16114 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 23:27:18.007599 16114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "c28c75dc9d52407d84e4af4541a024b0"
format_stamp: "Formatted at 2026-08-08 23:27:17 on dist-test-slave-6bbx"
server_key: "62011c01b30a711c0a9a3f4abb76e314"
server_key_iv: "55dc07c104cf654ae19ec11afcbb614e"
server_key_version: "encryptionkey@0"
I20260808 23:27:18.007839 16114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:18.013245 16114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:18.013927 16114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:18.014187 16114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:18.031903 16114 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:35533
I20260808 23:27:18.031946 16220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:35533 every 8 connection(s)
I20260808 23:27:18.033084 16114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:27:18.033474 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 16114
I20260808 23:27:18.033679 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:27:18.034502 9393 external_mini_cluster.cc:1428] Setting key 482b362b99205b3620b01560915cc93e
I20260808 23:27:18.037493 16221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:18.043604 16225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:18.036372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37002 (local address 127.9.44.126:35533)
0808 23:27:18.037452 (+ 1080us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:18.037467 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:18.037523 (+ 56us) server_negotiation.cc:407] Connection header received
0808 23:27:18.039738 (+ 2215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:18.039765 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:18.040248 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:18.040694 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:18.041667 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:18.041675 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:18.041690 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:27:18.041711 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:18.041814 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:27:18.041837 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:18.042054 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:18.042200 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 23:27:18.042466 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":349,"threads_started":1}
I20260808 23:27:18.049525 16221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c28c75dc9d52407d84e4af4541a024b0: Bootstrap starting.
I20260808 23:27:18.052474 16221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c28c75dc9d52407d84e4af4541a024b0: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:18.053385 16221 log.cc:824] T 00000000000000000000000000000000 P c28c75dc9d52407d84e4af4541a024b0: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:18.055883 16221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c28c75dc9d52407d84e4af4541a024b0: No bootstrap required, opened a new log
I20260808 23:27:18.062875 16221 raft_consensus.cc:348] T 00000000000000000000000000000000 P c28c75dc9d52407d84e4af4541a024b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c28c75dc9d52407d84e4af4541a024b0" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35533 } }
I20260808 23:27:18.063185 16221 raft_consensus.cc:374] T 00000000000000000000000000000000 P c28c75dc9d52407d84e4af4541a024b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:18.063258 16221 raft_consensus.cc:729] T 00000000000000000000000000000000 P c28c75dc9d52407d84e4af4541a024b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c28c75dc9d52407d84e4af4541a024b0, State: Initialized, Role: FOLLOWER
I20260808 23:27:18.063794 16221 consensus_queue.cc:260] T 00000000000000000000000000000000 P c28c75dc9d52407d84e4af4541a024b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c28c75dc9d52407d84e4af4541a024b0" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35533 } }
I20260808 23:27:18.064004 16221 raft_consensus.cc:388] T 00000000000000000000000000000000 P c28c75dc9d52407d84e4af4541a024b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:27:18.064106 16221 raft_consensus.cc:482] T 00000000000000000000000000000000 P c28c75dc9d52407d84e4af4541a024b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:27:18.064242 16221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c28c75dc9d52407d84e4af4541a024b0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:18.066260 16221 raft_consensus.cc:504] T 00000000000000000000000000000000 P c28c75dc9d52407d84e4af4541a024b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c28c75dc9d52407d84e4af4541a024b0" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35533 } }
I20260808 23:27:18.066702 16221 leader_election.cc:302] T 00000000000000000000000000000000 P c28c75dc9d52407d84e4af4541a024b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c28c75dc9d52407d84e4af4541a024b0; no voters:
I20260808 23:27:18.067199 16221 leader_election.cc:288] T 00000000000000000000000000000000 P c28c75dc9d52407d84e4af4541a024b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:27:18.067306 16227 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c28c75dc9d52407d84e4af4541a024b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:27:18.067987 16227 raft_consensus.cc:686] T 00000000000000000000000000000000 P c28c75dc9d52407d84e4af4541a024b0 [term 1 LEADER]: Becoming Leader. State: Replica: c28c75dc9d52407d84e4af4541a024b0, State: Running, Role: LEADER
I20260808 23:27:18.068488 16227 consensus_queue.cc:237] T 00000000000000000000000000000000 P c28c75dc9d52407d84e4af4541a024b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c28c75dc9d52407d84e4af4541a024b0" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35533 } }
I20260808 23:27:18.069128 16221 sys_catalog.cc:564] T 00000000000000000000000000000000 P c28c75dc9d52407d84e4af4541a024b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:27:18.070961 16228 sys_catalog.cc:455] T 00000000000000000000000000000000 P c28c75dc9d52407d84e4af4541a024b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c28c75dc9d52407d84e4af4541a024b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c28c75dc9d52407d84e4af4541a024b0" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35533 } } }
I20260808 23:27:18.071394 16228 sys_catalog.cc:458] T 00000000000000000000000000000000 P c28c75dc9d52407d84e4af4541a024b0 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:18.070959 16230 sys_catalog.cc:455] T 00000000000000000000000000000000 P c28c75dc9d52407d84e4af4541a024b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c28c75dc9d52407d84e4af4541a024b0. Latest consensus state: current_term: 1 leader_uuid: "c28c75dc9d52407d84e4af4541a024b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c28c75dc9d52407d84e4af4541a024b0" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35533 } } }
I20260808 23:27:18.072140 16230 sys_catalog.cc:458] T 00000000000000000000000000000000 P c28c75dc9d52407d84e4af4541a024b0 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:18.076593 16240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:27:18.081604 16240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:27:18.089818 16240 catalog_manager.cc:1347] Generated new cluster ID: 01546b6375d346a3b92103b01b366a26
I20260808 23:27:18.089911 16240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:27:18.101781 16240 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:27:18.102113 16240 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:27:18.108142 16240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c28c75dc9d52407d84e4af4541a024b0: Generated new TSK 0
I20260808 23:27:18.108783 16240 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:27:18.112697 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:0
--local_ip_for_outbound_sockets=127.9.44.65
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:35533
--builtin_ntp_servers=127.9.44.84:38701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 23:27:18.435083 16263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:18.435331 16263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:18.435432 16263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:18.442302 16263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:18.442394 16263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:18.442503 16263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:27:18.450934 16263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:38701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:35533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:18.452037 16263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:18.453711 16263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:18.465902 16272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:18.465858 16273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:18.465950 16277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:18.466679 16263 server_base.cc:1034] running on GCE node
I20260808 23:27:18.467257 16263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:18.468164 16263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:18.469363 16263 hybrid_clock.cc:648] HybridClock initialized: now 1786231638469309 us; error 46 us; skew 500 ppm
I20260808 23:27:18.469687 16263 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:27:18.471839 16263 webserver.cc:460] Webserver started at http://127.9.44.65:41829/ using document root <none> and password file <none>
I20260808 23:27:18.472479 16263 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:18.472577 16263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:18.472882 16263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:18.475550 16263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-0/data/instance:
uuid: "e651bc2108c446ac8525a42b7e1ae647"
format_stamp: "Formatted at 2026-08-08 23:27:18 on dist-test-slave-6bbx"
server_key: "6b4b61f8db27af1e52c8ea375d6bf45e"
server_key_iv: "2d864ba085c0b8c43213e5e8976ec5eb"
server_key_version: "encryptionkey@0"
I20260808 23:27:18.476375 16263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance:
uuid: "e651bc2108c446ac8525a42b7e1ae647"
format_stamp: "Formatted at 2026-08-08 23:27:18 on dist-test-slave-6bbx"
server_key: "6b4b61f8db27af1e52c8ea375d6bf45e"
server_key_iv: "2d864ba085c0b8c43213e5e8976ec5eb"
server_key_version: "encryptionkey@0"
I20260808 23:27:18.481168 16263 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 23:27:18.484386 16283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:18.485560 16263 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 23:27:18.485702 16263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "e651bc2108c446ac8525a42b7e1ae647"
format_stamp: "Formatted at 2026-08-08 23:27:18 on dist-test-slave-6bbx"
server_key: "6b4b61f8db27af1e52c8ea375d6bf45e"
server_key_iv: "2d864ba085c0b8c43213e5e8976ec5eb"
server_key_version: "encryptionkey@0"
I20260808 23:27:18.485906 16263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:18.491003 16263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:18.491739 16263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:18.491981 16263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:18.492822 16263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:18.494089 16263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:18.494184 16263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:18.494298 16263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:18.494369 16263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:18.522339 16263 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:34643
I20260808 23:27:18.522401 16452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:34643 every 8 connection(s)
I20260808 23:27:18.523440 16263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:27:18.530535 16225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:18.525742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:53061 (local address 127.9.44.126:35533)
0808 23:27:18.525954 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:18.525960 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:18.527176 (+ 1216us) server_negotiation.cc:407] Connection header received
0808 23:27:18.528763 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:18.528767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:18.528821 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:18.528911 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:18.529995 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:18.530000 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:18.530003 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:27:18.530019 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:18.530051 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:27:18.530063 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:18.530152 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:18.530278 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 23:27:18.530325 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 23:27:18.531342 16458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:18.526194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35533 (local address 127.9.44.65:53061)
0808 23:27:18.527046 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:18.527079 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:18.528544 (+ 1465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:18.529080 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:18.529087 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:18.529169 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:18.529732 (+ 563us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:27:18.529735 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:18.529769 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:27:18.529848 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:18.530134 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:18.530139 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:18.530144 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:27:18.530336 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 23:27:18.530509 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":269,"threads_started":1}
I20260808 23:27:18.532707 16454 heartbeater.cc:347] Connected to a master server at 127.9.44.126:35533
I20260808 23:27:18.532959 16454 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:18.533486 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 16263
I20260808 23:27:18.533501 16454 heartbeater.cc:511] Master 127.9.44.126:35533 requested a full tablet report, sending...
I20260808 23:27:18.533646 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance
I20260808 23:27:18.534307 9393 external_mini_cluster.cc:1428] Setting key 41614bd2f10d853478e2c01d7741de74
I20260808 23:27:18.535558 16167 ts_manager.cc:194] Registered new tserver with Master: e651bc2108c446ac8525a42b7e1ae647 (127.9.44.65:34643)
I20260808 23:27:18.537294 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:0
--local_ip_for_outbound_sockets=127.9.44.66
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:35533
--builtin_ntp_servers=127.9.44.84:38701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 23:27:18.853897 16459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:18.854154 16459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:18.854292 16459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:18.861373 16459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:18.861461 16459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:18.861600 16459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:27:18.870823 16459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:38701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:35533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:18.871989 16459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:18.873672 16459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:18.885807 16468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:18.885900 16469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:18.885807 16471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:18.887583 16459 server_base.cc:1034] running on GCE node
I20260808 23:27:18.888187 16459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:18.889055 16459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:18.890280 16459 hybrid_clock.cc:648] HybridClock initialized: now 1786231638890214 us; error 53 us; skew 500 ppm
I20260808 23:27:18.890596 16459 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:27:18.892712 16459 webserver.cc:460] Webserver started at http://127.9.44.66:33041/ using document root <none> and password file <none>
I20260808 23:27:18.893349 16459 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:18.893447 16459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:18.893743 16459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:18.896495 16459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-1/data/instance:
uuid: "621545c2942f441c92ef729c350402d1"
format_stamp: "Formatted at 2026-08-08 23:27:18 on dist-test-slave-6bbx"
server_key: "a1bffde291a5adb0e494ecc06f8a7161"
server_key_iv: "21a579c23cb7dcf7019b5d419c222e11"
server_key_version: "encryptionkey@0"
I20260808 23:27:18.897228 16459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance:
uuid: "621545c2942f441c92ef729c350402d1"
format_stamp: "Formatted at 2026-08-08 23:27:18 on dist-test-slave-6bbx"
server_key: "a1bffde291a5adb0e494ecc06f8a7161"
server_key_iv: "21a579c23cb7dcf7019b5d419c222e11"
server_key_version: "encryptionkey@0"
I20260808 23:27:18.902169 16459 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:27:18.905337 16478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:18.906520 16459 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 23:27:18.906665 16459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "621545c2942f441c92ef729c350402d1"
format_stamp: "Formatted at 2026-08-08 23:27:18 on dist-test-slave-6bbx"
server_key: "a1bffde291a5adb0e494ecc06f8a7161"
server_key_iv: "21a579c23cb7dcf7019b5d419c222e11"
server_key_version: "encryptionkey@0"
I20260808 23:27:18.906865 16459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:18.912173 16459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:18.912883 16459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:18.913118 16459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:18.913959 16459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:18.915220 16459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:18.915323 16459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:18.915426 16459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:18.915496 16459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:18.945209 16459 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:33969
I20260808 23:27:18.945276 16655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:33969 every 8 connection(s)
I20260808 23:27:18.946398 16459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:27:18.946795 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 16459
I20260808 23:27:18.946971 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance
I20260808 23:27:18.947539 9393 external_mini_cluster.cc:1428] Setting key 8b95d7c8bb8f879acebec6ea45a05b4b
I20260808 23:27:18.950664 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:0
--local_ip_for_outbound_sockets=127.9.44.67
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:35533
--builtin_ntp_servers=127.9.44.84:38701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 23:27:18.954506 16225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:18.949037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:42591 (local address 127.9.44.126:35533)
0808 23:27:18.949235 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:18.949242 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:18.950671 (+ 1429us) server_negotiation.cc:407] Connection header received
0808 23:27:18.952361 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:18.952367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:18.952430 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:18.952518 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:18.953800 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:18.953806 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:18.953811 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:27:18.953829 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:18.953862 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:27:18.953874 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:18.953973 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:18.954164 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 23:27:18.954231 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 23:27:18.955365 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:18.949558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35533 (local address 127.9.44.66:42591)
0808 23:27:18.950516 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:18.950552 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:18.952092 (+ 1540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:18.952757 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:18.952765 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:18.952849 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:18.953490 (+ 641us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:27:18.953493 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:18.953535 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:27:18.953632 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:18.953963 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:18.953973 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:18.953981 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 23:27:18.954259 (+ 278us) client_negotiation.cc:240] Negotiation successful
0808 23:27:18.954494 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":324,"threads_started":1}
I20260808 23:27:18.956579 16656 heartbeater.cc:347] Connected to a master server at 127.9.44.126:35533
I20260808 23:27:18.956872 16656 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:18.957414 16656 heartbeater.cc:511] Master 127.9.44.126:35533 requested a full tablet report, sending...
I20260808 23:27:18.958770 16167 ts_manager.cc:194] Registered new tserver with Master: 621545c2942f441c92ef729c350402d1 (127.9.44.66:33969)
W20260808 23:27:19.272141 16660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:19.272387 16660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:19.272480 16660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:19.279333 16660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:19.279422 16660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:19.279569 16660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:27:19.288293 16660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:38701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:35533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:19.289444 16660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:19.291148 16660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:19.303129 16673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:19.303272 16668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:19.303258 16669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:19.303978 16660 server_base.cc:1034] running on GCE node
I20260808 23:27:19.304549 16660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:19.305399 16660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:19.306589 16660 hybrid_clock.cc:648] HybridClock initialized: now 1786231639306545 us; error 36 us; skew 500 ppm
I20260808 23:27:19.306866 16660 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:27:19.309034 16660 webserver.cc:460] Webserver started at http://127.9.44.67:43353/ using document root <none> and password file <none>
I20260808 23:27:19.309659 16660 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:19.309755 16660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:19.310058 16660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:19.313138 16660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-2/data/instance:
uuid: "b308cbc8a1aa4bf49ee9f3507e9f42b3"
format_stamp: "Formatted at 2026-08-08 23:27:19 on dist-test-slave-6bbx"
server_key: "6674d51caeb2ff14da145c491c859bde"
server_key_iv: "fbfda3fe9db5377c38ba501e3cf71152"
server_key_version: "encryptionkey@0"
I20260808 23:27:19.313894 16660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance:
uuid: "b308cbc8a1aa4bf49ee9f3507e9f42b3"
format_stamp: "Formatted at 2026-08-08 23:27:19 on dist-test-slave-6bbx"
server_key: "6674d51caeb2ff14da145c491c859bde"
server_key_iv: "fbfda3fe9db5377c38ba501e3cf71152"
server_key_version: "encryptionkey@0"
I20260808 23:27:19.318681 16660 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 23:27:19.321878 16685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:19.323071 16660 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 23:27:19.323208 16660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "b308cbc8a1aa4bf49ee9f3507e9f42b3"
format_stamp: "Formatted at 2026-08-08 23:27:19 on dist-test-slave-6bbx"
server_key: "6674d51caeb2ff14da145c491c859bde"
server_key_iv: "fbfda3fe9db5377c38ba501e3cf71152"
server_key_version: "encryptionkey@0"
I20260808 23:27:19.323415 16660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:19.328691 16660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:19.329404 16660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:19.329640 16660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:19.330482 16660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:19.331794 16660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:19.331900 16660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:19.332010 16660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:19.332080 16660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:19.360648 16660 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:43447
I20260808 23:27:19.360713 16849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:43447 every 8 connection(s)
I20260808 23:27:19.361874 16660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:27:19.369175 16225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:19.364226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:59745 (local address 127.9.44.126:35533)
0808 23:27:19.364456 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:19.364462 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:19.365757 (+ 1295us) server_negotiation.cc:407] Connection header received
0808 23:27:19.367324 (+ 1567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:19.367328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:19.367379 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:19.367461 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:19.368627 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:19.368632 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:19.368635 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:27:19.368650 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:19.368682 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:27:19.368694 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:19.368795 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:19.368919 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 23:27:19.368967 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 23:27:19.370046 16853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:19.364723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35533 (local address 127.9.44.67:59745)
0808 23:27:19.365615 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:19.365656 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:19.367102 (+ 1446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:19.367641 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:19.367649 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:19.367772 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:19.368350 (+ 578us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:27:19.368353 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:19.368399 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:27:19.368481 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:19.368769 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:19.368775 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:19.368780 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:27:19.368976 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 23:27:19.369158 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":276,"threads_started":1}
I20260808 23:27:19.370959 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 16660
I20260808 23:27:19.371121 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance
I20260808 23:27:19.371428 16850 heartbeater.cc:347] Connected to a master server at 127.9.44.126:35533
I20260808 23:27:19.371628 9393 external_mini_cluster.cc:1428] Setting key 4c5eff368498d53ef03e766336afb1f4
I20260808 23:27:19.371764 16850 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:19.372411 16850 heartbeater.cc:511] Master 127.9.44.126:35533 requested a full tablet report, sending...
I20260808 23:27:19.373910 16166 ts_manager.cc:194] Registered new tserver with Master: b308cbc8a1aa4bf49ee9f3507e9f42b3 (127.9.44.67:43447)
I20260808 23:27:19.386469 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:27:19.392871 16225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:19.390683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37010 (local address 127.9.44.126:35533)
0808 23:27:19.390834 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:19.390838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:19.391335 (+ 497us) server_negotiation.cc:407] Connection header received
0808 23:27:19.391477 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:19.391480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:19.391526 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:19.391592 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:19.392399 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:19.392403 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:19.392406 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:27:19.392421 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:19.392450 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:27:19.392462 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:19.392552 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:19.392617 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 23:27:19.392660 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 23:27:19.396304 16166 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37010:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:27:19.402920 16166 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:27:19.419948 16871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:19.414569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:34643 (local address 127.0.0.1:53758)
0808 23:27:19.415445 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:19.415485 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:19.415632 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:19.417228 (+ 1596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:19.417237 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:19.417321 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:19.418045 (+ 724us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:27:19.418048 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:19.418071 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:27:19.418135 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:19.419246 (+ 1111us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:19.419258 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:19.419266 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 23:27:19.419420 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 23:27:19.419501 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":1319,"threads_started":1}
I20260808 23:27:19.420943 16870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:19.414725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53758 (local address 127.9.44.65:34643)
0808 23:27:19.416736 (+ 2011us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:19.416742 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:19.416765 (+ 23us) server_negotiation.cc:407] Connection header received
0808 23:27:19.416817 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:19.416822 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:19.416991 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:19.417749 (+ 758us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:19.418980 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:19.418990 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:19.419003 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:27:19.419019 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:19.419074 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:27:19.419094 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:19.420472 (+ 1378us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:19.420546 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 23:27:19.420653 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1881,"thread_start_us":1774,"threads_started":1}
I20260808 23:27:19.422868 16878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:19.416225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:33969 (local address 127.0.0.1:36638)
0808 23:27:19.420489 (+ 4264us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:19.420505 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:19.420679 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:19.421241 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:19.421245 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:19.421267 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:19.421896 (+ 629us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:27:19.421900 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:19.421916 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:27:19.421961 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:19.422314 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:19.422319 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:19.422323 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:27:19.422442 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 23:27:19.422489 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3770,"thread_start_us":3609,"threads_started":1}
I20260808 23:27:19.422879 16879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:19.416792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36638 (local address 127.9.44.66:33969)
0808 23:27:19.417360 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:19.417368 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:19.420587 (+ 3219us) server_negotiation.cc:407] Connection header received
0808 23:27:19.420856 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:19.420866 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:19.421083 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:19.421235 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:19.422131 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:19.422136 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:19.422146 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:27:19.422165 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:19.422201 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:27:19.422221 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:19.422365 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:19.422433 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 23:27:19.422515 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":264,"threads_started":1}
I20260808 23:27:19.424887 16355 tablet_service.cc:1467] Processing CreateTablet for tablet 180da81d7d5a4898a433661abb7d0dc5 (DEFAULT_TABLE table=test-table [id=7d2e3f7165ea46faaece177c42f36b6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:27:19.425468 16877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:19.415120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35626 (local address 127.9.44.67:43447)
0808 23:27:19.421404 (+ 6284us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:19.421412 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:19.421454 (+ 42us) server_negotiation.cc:407] Connection header received
0808 23:27:19.421515 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:19.421522 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:19.423297 (+ 1775us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:19.423498 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:19.424661 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:19.424669 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:19.424682 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:27:19.424701 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:19.424755 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:27:19.424774 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:19.424947 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:19.425011 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 23:27:19.425093 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6130,"thread_start_us":5815,"threads_started":1}
I20260808 23:27:19.425489 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:19.414726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:43447 (local address 127.0.0.1:35626)
0808 23:27:19.416186 (+ 1460us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:19.416199 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:19.416298 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:19.423495 (+ 7197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:19.423499 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:19.423520 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:19.424449 (+ 929us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:27:19.424452 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:19.424467 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:27:19.424510 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:19.424886 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:19.424891 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:19.424896 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:27:19.425016 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 23:27:19.425062 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1322,"thread_start_us":652,"threads_started":1}
I20260808 23:27:19.426551 16355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 180da81d7d5a4898a433661abb7d0dc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:19.426411 16555 tablet_service.cc:1467] Processing CreateTablet for tablet 180da81d7d5a4898a433661abb7d0dc5 (DEFAULT_TABLE table=test-table [id=7d2e3f7165ea46faaece177c42f36b6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:27:19.427835 16555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 180da81d7d5a4898a433661abb7d0dc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:19.429035 16765 tablet_service.cc:1467] Processing CreateTablet for tablet 180da81d7d5a4898a433661abb7d0dc5 (DEFAULT_TABLE table=test-table [id=7d2e3f7165ea46faaece177c42f36b6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:27:19.430356 16765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 180da81d7d5a4898a433661abb7d0dc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:19.437719 16884 tablet_bootstrap.cc:492] T 180da81d7d5a4898a433661abb7d0dc5 P e651bc2108c446ac8525a42b7e1ae647: Bootstrap starting.
I20260808 23:27:19.440711 16884 tablet_bootstrap.cc:654] T 180da81d7d5a4898a433661abb7d0dc5 P e651bc2108c446ac8525a42b7e1ae647: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:19.441674 16884 log.cc:824] T 180da81d7d5a4898a433661abb7d0dc5 P e651bc2108c446ac8525a42b7e1ae647: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:19.443989 16887 tablet_bootstrap.cc:492] T 180da81d7d5a4898a433661abb7d0dc5 P 621545c2942f441c92ef729c350402d1: Bootstrap starting.
I20260808 23:27:19.444811 16884 tablet_bootstrap.cc:492] T 180da81d7d5a4898a433661abb7d0dc5 P e651bc2108c446ac8525a42b7e1ae647: No bootstrap required, opened a new log
I20260808 23:27:19.445080 16884 ts_tablet_manager.cc:1397] T 180da81d7d5a4898a433661abb7d0dc5 P e651bc2108c446ac8525a42b7e1ae647: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 23:27:19.447428 16887 tablet_bootstrap.cc:654] T 180da81d7d5a4898a433661abb7d0dc5 P 621545c2942f441c92ef729c350402d1: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:19.448313 16892 tablet_bootstrap.cc:492] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3: Bootstrap starting.
I20260808 23:27:19.448576 16887 log.cc:824] T 180da81d7d5a4898a433661abb7d0dc5 P 621545c2942f441c92ef729c350402d1: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:19.451404 16887 tablet_bootstrap.cc:492] T 180da81d7d5a4898a433661abb7d0dc5 P 621545c2942f441c92ef729c350402d1: No bootstrap required, opened a new log
I20260808 23:27:19.451613 16892 tablet_bootstrap.cc:654] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:19.451715 16887 ts_tablet_manager.cc:1397] T 180da81d7d5a4898a433661abb7d0dc5 P 621545c2942f441c92ef729c350402d1: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260808 23:27:19.452778 16892 log.cc:824] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:19.453526 16884 raft_consensus.cc:348] T 180da81d7d5a4898a433661abb7d0dc5 P e651bc2108c446ac8525a42b7e1ae647 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b308cbc8a1aa4bf49ee9f3507e9f42b3" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43447 } } peers { permanent_uuid: "e651bc2108c446ac8525a42b7e1ae647" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 34643 } } peers { permanent_uuid: "621545c2942f441c92ef729c350402d1" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33969 } }
I20260808 23:27:19.453999 16884 raft_consensus.cc:374] T 180da81d7d5a4898a433661abb7d0dc5 P e651bc2108c446ac8525a42b7e1ae647 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:19.454111 16884 raft_consensus.cc:729] T 180da81d7d5a4898a433661abb7d0dc5 P e651bc2108c446ac8525a42b7e1ae647 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e651bc2108c446ac8525a42b7e1ae647, State: Initialized, Role: FOLLOWER
I20260808 23:27:19.454636 16884 consensus_queue.cc:260] T 180da81d7d5a4898a433661abb7d0dc5 P e651bc2108c446ac8525a42b7e1ae647 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b308cbc8a1aa4bf49ee9f3507e9f42b3" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43447 } } peers { permanent_uuid: "e651bc2108c446ac8525a42b7e1ae647" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 34643 } } peers { permanent_uuid: "621545c2942f441c92ef729c350402d1" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33969 } }
I20260808 23:27:19.455646 16892 tablet_bootstrap.cc:492] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3: No bootstrap required, opened a new log
I20260808 23:27:19.455634 16454 heartbeater.cc:503] Master 127.9.44.126:35533 was elected leader, sending a full tablet report...
I20260808 23:27:19.455917 16892 ts_tablet_manager.cc:1397] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.002s
I20260808 23:27:19.456266 16884 ts_tablet_manager.cc:1428] T 180da81d7d5a4898a433661abb7d0dc5 P e651bc2108c446ac8525a42b7e1ae647: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260808 23:27:19.461256 16887 raft_consensus.cc:348] T 180da81d7d5a4898a433661abb7d0dc5 P 621545c2942f441c92ef729c350402d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b308cbc8a1aa4bf49ee9f3507e9f42b3" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43447 } } peers { permanent_uuid: "e651bc2108c446ac8525a42b7e1ae647" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 34643 } } peers { permanent_uuid: "621545c2942f441c92ef729c350402d1" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33969 } }
I20260808 23:27:19.461643 16887 raft_consensus.cc:374] T 180da81d7d5a4898a433661abb7d0dc5 P 621545c2942f441c92ef729c350402d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:19.461745 16887 raft_consensus.cc:729] T 180da81d7d5a4898a433661abb7d0dc5 P 621545c2942f441c92ef729c350402d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 621545c2942f441c92ef729c350402d1, State: Initialized, Role: FOLLOWER
I20260808 23:27:19.462342 16887 consensus_queue.cc:260] T 180da81d7d5a4898a433661abb7d0dc5 P 621545c2942f441c92ef729c350402d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b308cbc8a1aa4bf49ee9f3507e9f42b3" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43447 } } peers { permanent_uuid: "e651bc2108c446ac8525a42b7e1ae647" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 34643 } } peers { permanent_uuid: "621545c2942f441c92ef729c350402d1" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33969 } }
I20260808 23:27:19.463243 16656 heartbeater.cc:503] Master 127.9.44.126:35533 was elected leader, sending a full tablet report...
I20260808 23:27:19.463881 16887 ts_tablet_manager.cc:1428] T 180da81d7d5a4898a433661abb7d0dc5 P 621545c2942f441c92ef729c350402d1: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260808 23:27:19.465608 16892 raft_consensus.cc:348] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b308cbc8a1aa4bf49ee9f3507e9f42b3" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43447 } } peers { permanent_uuid: "e651bc2108c446ac8525a42b7e1ae647" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 34643 } } peers { permanent_uuid: "621545c2942f441c92ef729c350402d1" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33969 } }
I20260808 23:27:19.465896 16892 raft_consensus.cc:374] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:19.465986 16892 raft_consensus.cc:729] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b308cbc8a1aa4bf49ee9f3507e9f42b3, State: Initialized, Role: FOLLOWER
I20260808 23:27:19.466476 16892 consensus_queue.cc:260] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b308cbc8a1aa4bf49ee9f3507e9f42b3" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43447 } } peers { permanent_uuid: "e651bc2108c446ac8525a42b7e1ae647" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 34643 } } peers { permanent_uuid: "621545c2942f441c92ef729c350402d1" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33969 } }
I20260808 23:27:19.467525 16850 heartbeater.cc:503] Master 127.9.44.126:35533 was elected leader, sending a full tablet report...
I20260808 23:27:19.468036 16892 ts_tablet_manager.cc:1428] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3: Time spent starting tablet: real 0.012s user 0.009s sys 0.003s
W20260808 23:27:19.525585 16457 tablet.cc:2369] T 180da81d7d5a4898a433661abb7d0dc5 P e651bc2108c446ac8525a42b7e1ae647: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:27:19.613426 16852 tablet.cc:2369] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:27:19.616799 16898 raft_consensus.cc:482] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:27:19.617038 16898 raft_consensus.cc:504] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b308cbc8a1aa4bf49ee9f3507e9f42b3" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43447 } } peers { permanent_uuid: "e651bc2108c446ac8525a42b7e1ae647" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 34643 } } peers { permanent_uuid: "621545c2942f441c92ef729c350402d1" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33969 } }
I20260808 23:27:19.618656 16898 leader_election.cc:288] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e651bc2108c446ac8525a42b7e1ae647 (127.9.44.65:34643), 621545c2942f441c92ef729c350402d1 (127.9.44.66:33969)
I20260808 23:27:19.621342 16853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:19.619051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:34643 (local address 127.9.44.67:38337)
0808 23:27:19.619347 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:19.619363 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:19.619465 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:19.619899 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:19.619904 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:19.619928 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:19.620566 (+ 638us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:27:19.620569 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:19.620587 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:27:19.620643 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:19.620955 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:19.620959 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:19.620964 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:27:19.621070 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 23:27:19.621123 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195}
I20260808 23:27:19.621649 16879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:19.619174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:57953 (local address 127.9.44.66:33969)
0808 23:27:19.619369 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:19.619374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:19.620014 (+ 640us) server_negotiation.cc:407] Connection header received
0808 23:27:19.620136 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:19.620140 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:19.620199 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:19.620302 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:19.621101 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:19.621105 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:19.621107 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:27:19.621121 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:19.621149 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:27:19.621159 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:19.621251 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:19.621364 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 23:27:19.621416 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 23:27:19.621649 16870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:19.619106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:38337 (local address 127.9.44.65:34643)
0808 23:27:19.619290 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:19.619296 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:19.619426 (+ 130us) server_negotiation.cc:407] Connection header received
0808 23:27:19.619635 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:19.619641 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:19.619731 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:19.619839 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:19.620785 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:19.620791 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:19.620796 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:27:19.620815 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:19.620854 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:27:19.620868 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:19.620956 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:19.621313 (+ 357us) server_negotiation.cc:299] Negotiation successful
0808 23:27:19.621374 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 23:27:19.621817 16901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:19.619122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:33969 (local address 127.9.44.67:57953)
0808 23:27:19.619908 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:19.619924 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:19.620033 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:19.620326 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:19.620329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:19.620348 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:19.620913 (+ 565us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:27:19.620917 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:19.620932 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:27:19.620972 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:19.621236 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:19.621241 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:19.621245 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 23:27:19.621374 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 23:27:19.621422 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":450,"threads_started":1}
I20260808 23:27:19.622594 16383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "180da81d7d5a4898a433661abb7d0dc5" candidate_uuid: "b308cbc8a1aa4bf49ee9f3507e9f42b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e651bc2108c446ac8525a42b7e1ae647" is_pre_election: true
I20260808 23:27:19.622695 16585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "180da81d7d5a4898a433661abb7d0dc5" candidate_uuid: "b308cbc8a1aa4bf49ee9f3507e9f42b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "621545c2942f441c92ef729c350402d1" is_pre_election: true
I20260808 23:27:19.623064 16383 raft_consensus.cc:2454] T 180da81d7d5a4898a433661abb7d0dc5 P e651bc2108c446ac8525a42b7e1ae647 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b308cbc8a1aa4bf49ee9f3507e9f42b3 in term 0.
I20260808 23:27:19.623101 16585 raft_consensus.cc:2454] T 180da81d7d5a4898a433661abb7d0dc5 P 621545c2942f441c92ef729c350402d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b308cbc8a1aa4bf49ee9f3507e9f42b3 in term 0.
I20260808 23:27:19.623735 16688 leader_election.cc:302] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 621545c2942f441c92ef729c350402d1, b308cbc8a1aa4bf49ee9f3507e9f42b3, e651bc2108c446ac8525a42b7e1ae647; no voters:
I20260808 23:27:19.624111 16898 raft_consensus.cc:2781] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:27:19.624267 16898 raft_consensus.cc:482] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:27:19.624368 16898 raft_consensus.cc:3037] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:19.626459 16898 raft_consensus.cc:504] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b308cbc8a1aa4bf49ee9f3507e9f42b3" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43447 } } peers { permanent_uuid: "e651bc2108c446ac8525a42b7e1ae647" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 34643 } } peers { permanent_uuid: "621545c2942f441c92ef729c350402d1" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33969 } }
I20260808 23:27:19.627017 16898 leader_election.cc:288] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3 [CANDIDATE]: Term 1 election: Requested vote from peers e651bc2108c446ac8525a42b7e1ae647 (127.9.44.65:34643), 621545c2942f441c92ef729c350402d1 (127.9.44.66:33969)
I20260808 23:27:19.627337 16383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "180da81d7d5a4898a433661abb7d0dc5" candidate_uuid: "b308cbc8a1aa4bf49ee9f3507e9f42b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e651bc2108c446ac8525a42b7e1ae647"
I20260808 23:27:19.627455 16585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "180da81d7d5a4898a433661abb7d0dc5" candidate_uuid: "b308cbc8a1aa4bf49ee9f3507e9f42b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "621545c2942f441c92ef729c350402d1"
I20260808 23:27:19.627506 16383 raft_consensus.cc:3037] T 180da81d7d5a4898a433661abb7d0dc5 P e651bc2108c446ac8525a42b7e1ae647 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:19.627633 16585 raft_consensus.cc:3037] T 180da81d7d5a4898a433661abb7d0dc5 P 621545c2942f441c92ef729c350402d1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:19.629738 16383 raft_consensus.cc:2454] T 180da81d7d5a4898a433661abb7d0dc5 P e651bc2108c446ac8525a42b7e1ae647 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b308cbc8a1aa4bf49ee9f3507e9f42b3 in term 1.
I20260808 23:27:19.629845 16585 raft_consensus.cc:2454] T 180da81d7d5a4898a433661abb7d0dc5 P 621545c2942f441c92ef729c350402d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b308cbc8a1aa4bf49ee9f3507e9f42b3 in term 1.
I20260808 23:27:19.630191 16687 leader_election.cc:302] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b308cbc8a1aa4bf49ee9f3507e9f42b3, e651bc2108c446ac8525a42b7e1ae647; no voters:
I20260808 23:27:19.630481 16898 raft_consensus.cc:2781] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:27:19.630841 16898 raft_consensus.cc:686] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3 [term 1 LEADER]: Becoming Leader. State: Replica: b308cbc8a1aa4bf49ee9f3507e9f42b3, State: Running, Role: LEADER
I20260808 23:27:19.631294 16898 consensus_queue.cc:237] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b308cbc8a1aa4bf49ee9f3507e9f42b3" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43447 } } peers { permanent_uuid: "e651bc2108c446ac8525a42b7e1ae647" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 34643 } } peers { permanent_uuid: "621545c2942f441c92ef729c350402d1" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33969 } }
I20260808 23:27:19.635365 16166 catalog_manager.cc:5495] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to b308cbc8a1aa4bf49ee9f3507e9f42b3 (127.9.44.67). New cstate: current_term: 1 leader_uuid: "b308cbc8a1aa4bf49ee9f3507e9f42b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b308cbc8a1aa4bf49ee9f3507e9f42b3" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e651bc2108c446ac8525a42b7e1ae647" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 34643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "621545c2942f441c92ef729c350402d1" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33969 } health_report { overall_health: UNKNOWN } } }
I20260808 23:27:19.658679 16877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:19.656956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35634 (local address 127.9.44.67:43447)
0808 23:27:19.657090 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:19.657095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:19.657254 (+ 159us) server_negotiation.cc:407] Connection header received
0808 23:27:19.657409 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:19.657412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:19.657481 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:19.657559 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:19.658276 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:19.658281 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:19.658284 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:27:19.658296 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:19.658324 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:27:19.658335 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:19.658426 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:19.658474 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 23:27:19.658515 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 23:27:19.665925 16585 raft_consensus.cc:1261] T 180da81d7d5a4898a433661abb7d0dc5 P 621545c2942f441c92ef729c350402d1 [term 1 FOLLOWER]: Refusing update from remote peer b308cbc8a1aa4bf49ee9f3507e9f42b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:27:19.665975 16383 raft_consensus.cc:1261] T 180da81d7d5a4898a433661abb7d0dc5 P e651bc2108c446ac8525a42b7e1ae647 [term 1 FOLLOWER]: Refusing update from remote peer b308cbc8a1aa4bf49ee9f3507e9f42b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:27:19.666781 16902 consensus_queue.cc:1035] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "621545c2942f441c92ef729c350402d1" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:27:19.667078 16898 consensus_queue.cc:1035] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e651bc2108c446ac8525a42b7e1ae647" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 34643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:27:19.677110 16906 mvcc.cc:205] Tried to move back new op lower bound from 7316404796063629312 to 7316404795933437952. Current Snapshot: MvccSnapshot[applied={T|T < 7316404796063629312}]
I20260808 23:27:19.692901 16167 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:27:19.693110 16167 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:27:19.696393 16167 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c28c75dc9d52407d84e4af4541a024b0: Sending DeleteTablet for 3 replicas of tablet 180da81d7d5a4898a433661abb7d0dc5
I20260808 23:27:19.697496 16765 tablet_service.cc:1514] Processing DeleteTablet for tablet 180da81d7d5a4898a433661abb7d0dc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:27:19 UTC) from {username='slave'} at 127.0.0.1:35626
I20260808 23:27:19.697731 16555 tablet_service.cc:1514] Processing DeleteTablet for tablet 180da81d7d5a4898a433661abb7d0dc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:27:19 UTC) from {username='slave'} at 127.0.0.1:36638
I20260808 23:27:19.697738 16355 tablet_service.cc:1514] Processing DeleteTablet for tablet 180da81d7d5a4898a433661abb7d0dc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:27:19 UTC) from {username='slave'} at 127.0.0.1:53758
I20260808 23:27:19.701547 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 16263
I20260808 23:27:19.708649 16921 tablet_replica.cc:330] T 180da81d7d5a4898a433661abb7d0dc5 P 621545c2942f441c92ef729c350402d1: stopping tablet replica
I20260808 23:27:19.708873 16919 tablet_replica.cc:330] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3: stopping tablet replica
I20260808 23:27:19.709059 16921 raft_consensus.cc:2229] T 180da81d7d5a4898a433661abb7d0dc5 P 621545c2942f441c92ef729c350402d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:27:19.709227 16919 raft_consensus.cc:2229] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:27:19.709913 16919 raft_consensus.cc:2258] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:27:19.709967 16920 tablet_replica.cc:330] T 180da81d7d5a4898a433661abb7d0dc5 P e651bc2108c446ac8525a42b7e1ae647: stopping tablet replica
I20260808 23:27:19.710695 16920 raft_consensus.cc:2229] T 180da81d7d5a4898a433661abb7d0dc5 P e651bc2108c446ac8525a42b7e1ae647 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:27:19.711149 16920 raft_consensus.cc:2258] T 180da81d7d5a4898a433661abb7d0dc5 P e651bc2108c446ac8525a42b7e1ae647 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:27:19.711548 16921 raft_consensus.cc:2258] T 180da81d7d5a4898a433661abb7d0dc5 P 621545c2942f441c92ef729c350402d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:27:19.712049 16919 ts_tablet_manager.cc:1905] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:27:19.713428 16920 ts_tablet_manager.cc:1905] T 180da81d7d5a4898a433661abb7d0dc5 P e651bc2108c446ac8525a42b7e1ae647: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:27:19.719143 16870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:19.702683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53764 (local address 127.9.44.65:34643)
0808 23:27:19.702856 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:19.702862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:19.716550 (+ 13688us) server_negotiation.cc:407] Connection header received
0808 23:27:19.716977 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:19.716983 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:19.717090 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:19.717169 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:19.718321 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:19.718331 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:19.718336 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:27:19.718356 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:19.718393 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:27:19.718412 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:19.718514 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:19.718757 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 23:27:19.718819 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 23:27:19.719177 16919 ts_tablet_manager.cc:1918] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:27:19.719316 16919 log.cc:1196] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/180da81d7d5a4898a433661abb7d0dc5
I20260808 23:27:19.719897 16919 ts_tablet_manager.cc:1939] T 180da81d7d5a4898a433661abb7d0dc5 P b308cbc8a1aa4bf49ee9f3507e9f42b3: Deleting consensus metadata
I20260808 23:27:19.719972 16920 ts_tablet_manager.cc:1918] T 180da81d7d5a4898a433661abb7d0dc5 P e651bc2108c446ac8525a42b7e1ae647: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:27:19.720113 16920 log.cc:1196] T 180da81d7d5a4898a433661abb7d0dc5 P e651bc2108c446ac8525a42b7e1ae647: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-0/wal/wals/180da81d7d5a4898a433661abb7d0dc5
I20260808 23:27:19.720412 16447 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:19.721973 16921 ts_tablet_manager.cc:1905] T 180da81d7d5a4898a433661abb7d0dc5 P 621545c2942f441c92ef729c350402d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:27:19.722044 16141 catalog_manager.cc:4841] TS b308cbc8a1aa4bf49ee9f3507e9f42b3 (127.9.44.67:43447): tablet 180da81d7d5a4898a433661abb7d0dc5 (table test-table [id=7d2e3f7165ea46faaece177c42f36b6e]) successfully deleted
I20260808 23:27:19.726809 16921 ts_tablet_manager.cc:1918] T 180da81d7d5a4898a433661abb7d0dc5 P 621545c2942f441c92ef729c350402d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:27:19.727083 16921 log.cc:1196] T 180da81d7d5a4898a433661abb7d0dc5 P 621545c2942f441c92ef729c350402d1: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/180da81d7d5a4898a433661abb7d0dc5
I20260808 23:27:19.728001 16921 ts_tablet_manager.cc:1939] T 180da81d7d5a4898a433661abb7d0dc5 P 621545c2942f441c92ef729c350402d1: Deleting consensus metadata
I20260808 23:27:19.730015 16141 catalog_manager.cc:4841] TS 621545c2942f441c92ef729c350402d1 (127.9.44.66:33969): tablet 180da81d7d5a4898a433661abb7d0dc5 (table test-table [id=7d2e3f7165ea46faaece177c42f36b6e]) successfully deleted
I20260808 23:27:19.811012 16920 ts_tablet_manager.cc:1939] T 180da81d7d5a4898a433661abb7d0dc5 P e651bc2108c446ac8525a42b7e1ae647: Deleting consensus metadata
I20260808 23:27:19.811750 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 16263
W20260808 23:27:19.833771 16140 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.44.65:34643 (error 108)
I20260808 23:27:19.833958 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 16459
I20260808 23:27:19.835697 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:19.835241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:34643 (local address 127.0.0.1:53780)
0808 23:27:19.835421 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:19.835533 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260808 23:27:19.835985 16140 catalog_manager.cc:4584] TS e651bc2108c446ac8525a42b7e1ae647 (127.9.44.65:34643): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 180da81d7d5a4898a433661abb7d0dc5: Network error: Client connection negotiation failed: client connection to 127.9.44.65:34643: connect: Connection refused (error 111)
I20260808 23:27:19.837267 16879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:19.834793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36646 (local address 127.9.44.66:33969)
0808 23:27:19.834978 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:19.834984 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:19.835410 (+ 426us) server_negotiation.cc:407] Connection header received
0808 23:27:19.835490 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:19.835496 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:19.835609 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:19.835745 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:19.836661 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:19.836667 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:19.836672 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:27:19.836691 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:19.836731 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:27:19.836747 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:19.836879 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:19.836924 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 23:27:19.836981 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:27:19.837936 16649 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:19.889683 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:19.889124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:34643 (local address 127.0.0.1:53786)
0808 23:27:19.889370 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:19.889493 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260808 23:27:19.890831 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:19.890410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:34643 (local address 127.0.0.1:53798)
0808 23:27:19.890572 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:19.890670 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 23:27:19.923692 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 16459
I20260808 23:27:19.951416 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:19.950959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:34643 (local address 127.0.0.1:53810)
0808 23:27:19.951162 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:19.951250 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 23:27:19.952481 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:19.952138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:34643 (local address 127.0.0.1:53816)
0808 23:27:19.952285 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:19.952362 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 23:27:19.953632 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 16660
I20260808 23:27:19.956399 16877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:19.954325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35638 (local address 127.9.44.67:43447)
0808 23:27:19.954488 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:19.954492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:19.954510 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:27:19.954696 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:19.954700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:19.954750 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:19.954833 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:19.955771 (+ 938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:19.955775 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:19.955778 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:27:19.955792 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:19.955824 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:27:19.955836 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:19.955958 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:19.956072 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 23:27:19.956116 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 23:27:19.957094 16845 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:20.033087 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:20.032533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:34643 (local address 127.0.0.1:53832)
0808 23:27:20.032791 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:20.032904 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 23:27:20.034281 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:20.033828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:34643 (local address 127.0.0.1:53846)
0808 23:27:20.034043 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:20.034129 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260808 23:27:20.076655 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 16660
I20260808 23:27:20.104475 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 16114
I20260808 23:27:20.105727 16213 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:20.209619 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:20.208309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:34643 (local address 127.0.0.1:53862)
0808 23:27:20.209148 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:20.209278 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":690}
I20260808 23:27:20.210039 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 16114
2026-08-08T23:27:20Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2635 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 23:27:20.254678 9393 mini_oidc.cc:152] Starting JWKS server
I20260808 23:27:20.257546 9393 webserver.cc:460] Webserver started at https://127.0.0.1:45365/ using document root <none> and password file <none>
I20260808 23:27:20.257896 9393 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 23:27:20.258486 9393 webserver.cc:460] Webserver started at http://127.0.0.1:38807/ using document root <none> and password file <none>
2026-08-08T23:27:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:27:20Z Disabled control of system clock
I20260808 23:27:20.287413 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:35261
--webserver_interface=127.9.44.126
--webserver_port=0
--builtin_ntp_servers=127.9.44.84:45909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:35261
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45365/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786231597224934-9393-0/testchainca.cert with env {}
W20260808 23:27:20.594599 16941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:20.594878 16941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:20.595011 16941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:20.601979 16941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:27:20.602098 16941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:20.602173 16941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:20.602264 16941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:27:20.602324 16941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 23:27:20.602409 16941 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 23:27:20.602464 16941 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45365/jwks.json
I20260808 23:27:20.611805 16941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:45909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:35261
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:35261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45365/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786231597224934-9393-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:20.612910 16941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:20.614105 16941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:20.624892 16952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:20.624962 16948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:20.624933 16950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:20.625814 16941 server_base.cc:1034] running on GCE node
I20260808 23:27:20.626494 16941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:20.627727 16941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:20.628943 16941 hybrid_clock.cc:648] HybridClock initialized: now 1786231640628889 us; error 50 us; skew 500 ppm
I20260808 23:27:20.629261 16941 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:27:20.631304 16941 webserver.cc:460] Webserver started at http://127.9.44.126:35747/ using document root <none> and password file <none>
I20260808 23:27:20.631974 16941 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:20.632078 16941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:20.632357 16941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:20.634990 16941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "e0c2047136574dd2a2639913ad4c14f6"
format_stamp: "Formatted at 2026-08-08 23:27:20 on dist-test-slave-6bbx"
server_key: "4029b14553e61ba03e4ee91d8b7a7cd1"
server_key_iv: "d030c82db17be68863fa3178917e5de6"
server_key_version: "encryptionkey@0"
I20260808 23:27:20.635804 16941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "e0c2047136574dd2a2639913ad4c14f6"
format_stamp: "Formatted at 2026-08-08 23:27:20 on dist-test-slave-6bbx"
server_key: "4029b14553e61ba03e4ee91d8b7a7cd1"
server_key_iv: "d030c82db17be68863fa3178917e5de6"
server_key_version: "encryptionkey@0"
I20260808 23:27:20.640380 16941 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 23:27:20.643509 16958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:20.644714 16941 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 23:27:20.644866 16941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "e0c2047136574dd2a2639913ad4c14f6"
format_stamp: "Formatted at 2026-08-08 23:27:20 on dist-test-slave-6bbx"
server_key: "4029b14553e61ba03e4ee91d8b7a7cd1"
server_key_iv: "d030c82db17be68863fa3178917e5de6"
server_key_version: "encryptionkey@0"
I20260808 23:27:20.645047 16941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:20.659430 16941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:20.660152 16941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:20.660395 16941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:20.676796 16941 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:35261
I20260808 23:27:20.676815 17033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:35261 every 8 connection(s)
I20260808 23:27:20.677956 16941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:27:20.681393 17034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:20.681915 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 16941
I20260808 23:27:20.682145 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:27:20.682967 9393 external_mini_cluster.cc:1428] Setting key 6a039b6f79cc318a1464c337a15056fb
I20260808 23:27:20.694604 17034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0c2047136574dd2a2639913ad4c14f6: Bootstrap starting.
I20260808 23:27:20.696120 17038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:20.684768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33816 (local address 127.9.44.126:35261)
0808 23:27:20.685673 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:20.685682 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:20.685734 (+ 52us) server_negotiation.cc:407] Connection header received
0808 23:27:20.687536 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:20.687560 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:20.688025 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:20.688339 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:20.688950 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:20.690146 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:20.692009 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:20.692656 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:20.692776 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:20.694001 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:20.694008 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:20.694020 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:27:20.694511 (+ 491us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:20.694587 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:27:20.694611 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:20.694701 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:20.694930 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 23:27:20.695201 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":302,"threads_started":1}
I20260808 23:27:20.697930 17034 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0c2047136574dd2a2639913ad4c14f6: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:20.699069 17034 log.cc:824] T 00000000000000000000000000000000 P e0c2047136574dd2a2639913ad4c14f6: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:20.699759 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:27:20.701678 17034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0c2047136574dd2a2639913ad4c14f6: No bootstrap required, opened a new log
I20260808 23:27:20.708734 17034 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0c2047136574dd2a2639913ad4c14f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c2047136574dd2a2639913ad4c14f6" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35261 } }
I20260808 23:27:20.709041 17034 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0c2047136574dd2a2639913ad4c14f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:20.709112 17034 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0c2047136574dd2a2639913ad4c14f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0c2047136574dd2a2639913ad4c14f6, State: Initialized, Role: FOLLOWER
I20260808 23:27:20.709605 17034 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0c2047136574dd2a2639913ad4c14f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c2047136574dd2a2639913ad4c14f6" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35261 } }
I20260808 23:27:20.709806 17034 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0c2047136574dd2a2639913ad4c14f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:27:20.709903 17034 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0c2047136574dd2a2639913ad4c14f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:27:20.710028 17034 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0c2047136574dd2a2639913ad4c14f6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:20.712036 17034 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0c2047136574dd2a2639913ad4c14f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c2047136574dd2a2639913ad4c14f6" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35261 } }
I20260808 23:27:20.712498 17034 leader_election.cc:302] T 00000000000000000000000000000000 P e0c2047136574dd2a2639913ad4c14f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0c2047136574dd2a2639913ad4c14f6; no voters:
I20260808 23:27:20.712972 17034 leader_election.cc:288] T 00000000000000000000000000000000 P e0c2047136574dd2a2639913ad4c14f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:27:20.713122 17041 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0c2047136574dd2a2639913ad4c14f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:27:20.713850 17041 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0c2047136574dd2a2639913ad4c14f6 [term 1 LEADER]: Becoming Leader. State: Replica: e0c2047136574dd2a2639913ad4c14f6, State: Running, Role: LEADER
I20260808 23:27:20.714401 17041 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0c2047136574dd2a2639913ad4c14f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c2047136574dd2a2639913ad4c14f6" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35261 } }
I20260808 23:27:20.714846 17034 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0c2047136574dd2a2639913ad4c14f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:27:20.717602 17047 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c2047136574dd2a2639913ad4c14f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0c2047136574dd2a2639913ad4c14f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c2047136574dd2a2639913ad4c14f6" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35261 } } }
I20260808 23:27:20.717568 17044 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c2047136574dd2a2639913ad4c14f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0c2047136574dd2a2639913ad4c14f6. Latest consensus state: current_term: 1 leader_uuid: "e0c2047136574dd2a2639913ad4c14f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c2047136574dd2a2639913ad4c14f6" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35261 } } }
I20260808 23:27:20.718073 17044 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c2047136574dd2a2639913ad4c14f6 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:20.718073 17047 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c2047136574dd2a2639913ad4c14f6 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:20.719168 17056 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:27:20.724280 17056 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:27:20.733088 17056 catalog_manager.cc:1347] Generated new cluster ID: 79ae83386ade46d6af543b0f1796dcd9
I20260808 23:27:20.733182 17056 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:27:20.747390 17056 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:27:20.748803 17056 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:27:20.753839 17056 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0c2047136574dd2a2639913ad4c14f6: Generated new TSK 0
I20260808 23:27:20.754521 17056 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 23:27:20.793138 17081 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 23:27:20.793316 17081 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 23:27:20.794472 17038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:20.773007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33830 (local address 127.9.44.126:35261)
0808 23:27:20.773202 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:20.773209 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:20.773756 (+ 547us) server_negotiation.cc:407] Connection header received
0808 23:27:20.774041 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:20.774047 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:20.774129 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:20.774276 (+ 147us) server_negotiation.cc:226] Negotiated authn=JWT
0808 23:27:20.774788 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:20.775949 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:20.777268 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:20.777968 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:20.778099 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:20.778560 (+ 461us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 23:27:20.793861 (+ 15301us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45365/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 23:27:20.794183 (+ 322us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33830: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45365/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 23:27:20.794593 17038 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33830: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45365/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 23:27:20.795965 17079 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:20.773013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35261 (local address 127.0.0.1:33830)
0808 23:27:20.773662 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:20.773677 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:20.773810 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:20.774325 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:20.774328 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:20.774338 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0808 23:27:20.774628 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:20.774636 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:20.776075 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:20.776079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:20.777133 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:20.777144 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:20.778108 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:20.778112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:20.778236 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:20.778292 (+ 56us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 23:27:20.794086 (+ 15794us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45365/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 23:27:20.794318 (+ 232us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.44.126:35261: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45365/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":339,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.9.44.126:35261: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45365/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 23:27:20.797458 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 16941
I20260808 23:27:20.798435 17023 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:20.889211 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 16941
2026-08-08T23:27:20Z chronyd exiting
I20260808 23:27:20.942354 9393 test_util.cc:175] -----------------------------------------------
I20260808 23:27:20.942497 9393 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786231597224934-9393-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (692 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 23:27:20.946106 9393 mini_oidc.cc:152] Starting JWKS server
I20260808 23:27:20.947839 9393 webserver.cc:460] Webserver started at https://127.0.0.1:34295/ using document root <none> and password file <none>
I20260808 23:27:20.948050 9393 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 23:27:20.948676 9393 webserver.cc:460] Webserver started at http://127.0.0.1:34939/ using document root <none> and password file <none>
2026-08-08T23:27:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:27:20Z Disabled control of system clock
I20260808 23:27:20.977665 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:38093
--webserver_interface=127.9.44.126
--webserver_port=0
--builtin_ntp_servers=127.9.44.84:34011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:38093
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34295/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786231597224934-9393-0/testchainca.cert with env {}
W20260808 23:27:21.300563 17089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:21.300848 17089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:21.300979 17089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:21.307940 17089 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:27:21.308048 17089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:21.308115 17089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:21.308200 17089 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:27:21.308259 17089 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 23:27:21.308343 17089 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 23:27:21.308394 17089 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34295/jwks.json
I20260808 23:27:21.317757 17089 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:34011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:38093
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:38093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34295/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786231597224934-9393-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:21.318897 17089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:21.320159 17089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:21.330920 17098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:21.330952 17101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:21.331089 17096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:21.331890 17089 server_base.cc:1034] running on GCE node
I20260808 23:27:21.332640 17089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:21.333884 17089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:21.335076 17089 hybrid_clock.cc:648] HybridClock initialized: now 1786231641335033 us; error 36 us; skew 500 ppm
I20260808 23:27:21.335386 17089 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:27:21.337468 17089 webserver.cc:460] Webserver started at http://127.9.44.126:37865/ using document root <none> and password file <none>
I20260808 23:27:21.338090 17089 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:21.338202 17089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:21.338503 17089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:21.341240 17089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "2d23c35834c34c93b43d4a8e36d7b9a0"
format_stamp: "Formatted at 2026-08-08 23:27:21 on dist-test-slave-6bbx"
server_key: "4017ae83bd459c58c1dba75bf5f7e641"
server_key_iv: "26af9105ee8ccb2c76346fce0e195704"
server_key_version: "encryptionkey@0"
I20260808 23:27:21.342000 17089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "2d23c35834c34c93b43d4a8e36d7b9a0"
format_stamp: "Formatted at 2026-08-08 23:27:21 on dist-test-slave-6bbx"
server_key: "4017ae83bd459c58c1dba75bf5f7e641"
server_key_iv: "26af9105ee8ccb2c76346fce0e195704"
server_key_version: "encryptionkey@0"
I20260808 23:27:21.346736 17089 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260808 23:27:21.349963 17111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:21.351146 17089 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 23:27:21.351296 17089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "2d23c35834c34c93b43d4a8e36d7b9a0"
format_stamp: "Formatted at 2026-08-08 23:27:21 on dist-test-slave-6bbx"
server_key: "4017ae83bd459c58c1dba75bf5f7e641"
server_key_iv: "26af9105ee8ccb2c76346fce0e195704"
server_key_version: "encryptionkey@0"
I20260808 23:27:21.351485 17089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:21.364858 17089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:21.365538 17089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:21.365785 17089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:21.382501 17089 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:38093
I20260808 23:27:21.382520 17194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:38093 every 8 connection(s)
I20260808 23:27:21.383725 17089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:27:21.387122 17195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:21.393425 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 17089
I20260808 23:27:21.393641 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:27:21.394433 9393 external_mini_cluster.cc:1428] Setting key 6a3d84a9976fb672ebf18d71dfddcc6b
I20260808 23:27:21.397722 17195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d23c35834c34c93b43d4a8e36d7b9a0: Bootstrap starting.
I20260808 23:27:21.400866 17195 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d23c35834c34c93b43d4a8e36d7b9a0: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:21.401952 17195 log.cc:824] T 00000000000000000000000000000000 P 2d23c35834c34c93b43d4a8e36d7b9a0: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:21.404800 17195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d23c35834c34c93b43d4a8e36d7b9a0: No bootstrap required, opened a new log
I20260808 23:27:21.407130 17206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:21.396260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57096 (local address 127.9.44.126:38093)
0808 23:27:21.397144 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:21.397154 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:21.397193 (+ 39us) server_negotiation.cc:407] Connection header received
0808 23:27:21.398816 (+ 1623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:21.398838 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:21.399276 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:21.399579 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:21.400177 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:21.401457 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:21.403034 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:21.403706 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:21.403850 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:21.404984 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:21.404991 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:21.405005 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:27:21.405524 (+ 519us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:21.405597 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:27:21.405618 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:21.405705 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:21.405923 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 23:27:21.406217 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":319,"threads_started":1}
I20260808 23:27:21.410601 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:27:21.412968 17195 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d23c35834c34c93b43d4a8e36d7b9a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d23c35834c34c93b43d4a8e36d7b9a0" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 38093 } }
I20260808 23:27:21.413395 17195 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d23c35834c34c93b43d4a8e36d7b9a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:21.413501 17195 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d23c35834c34c93b43d4a8e36d7b9a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d23c35834c34c93b43d4a8e36d7b9a0, State: Initialized, Role: FOLLOWER
I20260808 23:27:21.414053 17195 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d23c35834c34c93b43d4a8e36d7b9a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d23c35834c34c93b43d4a8e36d7b9a0" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 38093 } }
I20260808 23:27:21.414296 17195 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d23c35834c34c93b43d4a8e36d7b9a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:27:21.414438 17195 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d23c35834c34c93b43d4a8e36d7b9a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:27:21.414606 17195 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d23c35834c34c93b43d4a8e36d7b9a0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:21.417040 17195 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d23c35834c34c93b43d4a8e36d7b9a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d23c35834c34c93b43d4a8e36d7b9a0" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 38093 } }
I20260808 23:27:21.417449 17195 leader_election.cc:302] T 00000000000000000000000000000000 P 2d23c35834c34c93b43d4a8e36d7b9a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d23c35834c34c93b43d4a8e36d7b9a0; no voters:
I20260808 23:27:21.417991 17195 leader_election.cc:288] T 00000000000000000000000000000000 P 2d23c35834c34c93b43d4a8e36d7b9a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:27:21.418088 17209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d23c35834c34c93b43d4a8e36d7b9a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:27:21.419015 17209 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d23c35834c34c93b43d4a8e36d7b9a0 [term 1 LEADER]: Becoming Leader. State: Replica: 2d23c35834c34c93b43d4a8e36d7b9a0, State: Running, Role: LEADER
I20260808 23:27:21.419464 17209 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d23c35834c34c93b43d4a8e36d7b9a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d23c35834c34c93b43d4a8e36d7b9a0" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 38093 } }
I20260808 23:27:21.419937 17195 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d23c35834c34c93b43d4a8e36d7b9a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:27:21.422241 17211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d23c35834c34c93b43d4a8e36d7b9a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d23c35834c34c93b43d4a8e36d7b9a0. Latest consensus state: current_term: 1 leader_uuid: "2d23c35834c34c93b43d4a8e36d7b9a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d23c35834c34c93b43d4a8e36d7b9a0" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 38093 } } }
I20260808 23:27:21.422799 17210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d23c35834c34c93b43d4a8e36d7b9a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d23c35834c34c93b43d4a8e36d7b9a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d23c35834c34c93b43d4a8e36d7b9a0" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 38093 } } }
I20260808 23:27:21.423038 17211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d23c35834c34c93b43d4a8e36d7b9a0 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:21.423094 17210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d23c35834c34c93b43d4a8e36d7b9a0 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:21.424994 17223 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:27:21.429720 17223 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:27:21.437986 17223 catalog_manager.cc:1347] Generated new cluster ID: 417f01d621bd4bf588e84db4214fe757
I20260808 23:27:21.438081 17223 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:27:21.449787 17223 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:27:21.450862 17223 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:27:21.461017 17223 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d23c35834c34c93b43d4a8e36d7b9a0: Generated new TSK 0
I20260808 23:27:21.461649 17223 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 23:27:21.499538 17262 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 23:27:21.499683 17262 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 23:27:21.500455 17206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:21.473931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57112 (local address 127.9.44.126:38093)
0808 23:27:21.474106 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:21.474111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:21.474443 (+ 332us) server_negotiation.cc:407] Connection header received
0808 23:27:21.474679 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:21.474682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:21.474747 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:21.474868 (+ 121us) server_negotiation.cc:226] Negotiated authn=JWT
0808 23:27:21.475373 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:21.476116 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:21.478145 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:21.478599 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:21.478684 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:21.479218 (+ 534us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 23:27:21.499979 (+ 20761us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34295/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 23:27:21.500262 (+ 283us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57112: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34295/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":65}
W20260808 23:27:21.500560 17206 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57112: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34295/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 23:27:21.500818 17258 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:21.473721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:38093 (local address 127.0.0.1:57112)
0808 23:27:21.474341 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:21.474360 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:21.474511 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:21.474906 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:21.474912 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:21.474924 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0808 23:27:21.475222 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:21.475235 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:21.476250 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:21.476256 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:21.478010 (+ 1754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:21.478026 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:21.478772 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:21.478779 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:21.478968 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:21.479012 (+ 44us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 23:27:21.500179 (+ 21167us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34295/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 23:27:21.500420 (+ 241us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.44.126:38093: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34295/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":323,"threads_started":1}
I20260808 23:27:21.501770 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 17089
I20260808 23:27:21.502835 17188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:21.590909 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 17089
2026-08-08T23:27:21Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (703 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 23:27:21.649850 9393 mini_oidc.cc:152] Starting JWKS server
I20260808 23:27:21.651610 9393 webserver.cc:460] Webserver started at https://127.0.0.1:41763/ using document root <none> and password file <none>
I20260808 23:27:21.651860 9393 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 23:27:21.652429 9393 webserver.cc:460] Webserver started at http://127.0.0.1:45047/ using document root <none> and password file <none>
2026-08-08T23:27:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:27:21Z Disabled control of system clock
I20260808 23:27:21.681217 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:46205
--webserver_interface=127.9.44.126
--webserver_port=0
--builtin_ntp_servers=127.9.44.84:34255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:46205
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41763/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 23:27:21.993045 17269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:21.993340 17269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:21.993477 17269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:22.000491 17269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:27:22.000607 17269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:22.000671 17269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:22.000752 17269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:27:22.000819 17269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 23:27:22.000905 17269 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 23:27:22.000959 17269 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41763/jwks.json
I20260808 23:27:22.009613 17269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:34255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:46205
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:46205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41763/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:22.010731 17269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:22.012449 17269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:22.023495 17277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:22.023535 17276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:22.023556 17280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:22.024271 17269 server_base.cc:1034] running on GCE node
I20260808 23:27:22.024974 17269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:22.026233 17269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:22.027420 17269 hybrid_clock.cc:648] HybridClock initialized: now 1786231642027380 us; error 32 us; skew 500 ppm
I20260808 23:27:22.027761 17269 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:27:22.029809 17269 webserver.cc:460] Webserver started at http://127.9.44.126:41613/ using document root <none> and password file <none>
I20260808 23:27:22.030435 17269 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:22.030525 17269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:22.030830 17269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:22.034255 17269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "4472c26366f34778b9bf93076b30a35a"
format_stamp: "Formatted at 2026-08-08 23:27:22 on dist-test-slave-6bbx"
server_key: "aea430a536d91730ad874bd53537fab2"
server_key_iv: "761726daf0981b2aaa0bac6ab97283e3"
server_key_version: "encryptionkey@0"
I20260808 23:27:22.035087 17269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "4472c26366f34778b9bf93076b30a35a"
format_stamp: "Formatted at 2026-08-08 23:27:22 on dist-test-slave-6bbx"
server_key: "aea430a536d91730ad874bd53537fab2"
server_key_iv: "761726daf0981b2aaa0bac6ab97283e3"
server_key_version: "encryptionkey@0"
I20260808 23:27:22.039856 17269 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260808 23:27:22.043040 17296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:22.044317 17269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:27:22.044481 17269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "4472c26366f34778b9bf93076b30a35a"
format_stamp: "Formatted at 2026-08-08 23:27:22 on dist-test-slave-6bbx"
server_key: "aea430a536d91730ad874bd53537fab2"
server_key_iv: "761726daf0981b2aaa0bac6ab97283e3"
server_key_version: "encryptionkey@0"
I20260808 23:27:22.044668 17269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:22.064342 17269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:22.065026 17269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:22.065279 17269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:22.081353 17269 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:46205
I20260808 23:27:22.081387 17379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:46205 every 8 connection(s)
I20260808 23:27:22.082584 17269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:27:22.086169 17384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:22.086193 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 17269
I20260808 23:27:22.086387 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:27:22.087002 9393 external_mini_cluster.cc:1428] Setting key 848e1a8f1cf33d1a87ad61ff1f1dd098
I20260808 23:27:22.100190 17384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4472c26366f34778b9bf93076b30a35a: Bootstrap starting.
I20260808 23:27:22.102021 17387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:22.088722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47836 (local address 127.9.44.126:46205)
0808 23:27:22.089870 (+ 1148us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:22.089886 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:22.089946 (+ 60us) server_negotiation.cc:407] Connection header received
0808 23:27:22.092535 (+ 2589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:22.092564 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:22.093140 (+ 576us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:22.093577 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:22.094092 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:22.095947 (+ 1855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:22.097195 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:22.098240 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:22.098410 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:22.099372 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:22.099383 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:22.099399 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:27:22.100096 (+ 697us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:22.100194 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:27:22.100221 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:22.100332 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:22.100515 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 23:27:22.100805 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":367,"threads_started":1}
I20260808 23:27:22.103899 17384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4472c26366f34778b9bf93076b30a35a: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:22.105070 17384 log.cc:824] T 00000000000000000000000000000000 P 4472c26366f34778b9bf93076b30a35a: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:22.105715 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:27:22.107656 17384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4472c26366f34778b9bf93076b30a35a: No bootstrap required, opened a new log
I20260808 23:27:22.115034 17384 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4472c26366f34778b9bf93076b30a35a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4472c26366f34778b9bf93076b30a35a" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 46205 } }
I20260808 23:27:22.115325 17384 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4472c26366f34778b9bf93076b30a35a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:22.115410 17384 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4472c26366f34778b9bf93076b30a35a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4472c26366f34778b9bf93076b30a35a, State: Initialized, Role: FOLLOWER
I20260808 23:27:22.115918 17384 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4472c26366f34778b9bf93076b30a35a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4472c26366f34778b9bf93076b30a35a" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 46205 } }
I20260808 23:27:22.116129 17384 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4472c26366f34778b9bf93076b30a35a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:27:22.116230 17384 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4472c26366f34778b9bf93076b30a35a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:27:22.116350 17384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4472c26366f34778b9bf93076b30a35a [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:22.118326 17384 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4472c26366f34778b9bf93076b30a35a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4472c26366f34778b9bf93076b30a35a" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 46205 } }
I20260808 23:27:22.118882 17384 leader_election.cc:302] T 00000000000000000000000000000000 P 4472c26366f34778b9bf93076b30a35a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4472c26366f34778b9bf93076b30a35a; no voters:
I20260808 23:27:22.119468 17384 leader_election.cc:288] T 00000000000000000000000000000000 P 4472c26366f34778b9bf93076b30a35a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:27:22.119567 17392 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4472c26366f34778b9bf93076b30a35a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:27:22.120237 17392 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4472c26366f34778b9bf93076b30a35a [term 1 LEADER]: Becoming Leader. State: Replica: 4472c26366f34778b9bf93076b30a35a, State: Running, Role: LEADER
I20260808 23:27:22.120755 17392 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4472c26366f34778b9bf93076b30a35a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4472c26366f34778b9bf93076b30a35a" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 46205 } }
I20260808 23:27:22.121299 17384 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4472c26366f34778b9bf93076b30a35a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:27:22.123436 17394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4472c26366f34778b9bf93076b30a35a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4472c26366f34778b9bf93076b30a35a. Latest consensus state: current_term: 1 leader_uuid: "4472c26366f34778b9bf93076b30a35a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4472c26366f34778b9bf93076b30a35a" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 46205 } } }
I20260808 23:27:22.123494 17393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4472c26366f34778b9bf93076b30a35a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4472c26366f34778b9bf93076b30a35a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4472c26366f34778b9bf93076b30a35a" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 46205 } } }
I20260808 23:27:22.123898 17394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4472c26366f34778b9bf93076b30a35a [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:22.123909 17393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4472c26366f34778b9bf93076b30a35a [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:22.125756 17402 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:27:22.129799 17402 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:27:22.137480 17402 catalog_manager.cc:1347] Generated new cluster ID: 1e657419f43c4ba8a6136c0b1553aec1
I20260808 23:27:22.137578 17402 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:27:22.153636 17402 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:27:22.154729 17402 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:27:22.173395 17402 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4472c26366f34778b9bf93076b30a35a: Generated new TSK 0
I20260808 23:27:22.174110 17402 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:27:22.200572 17387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:22.192353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47842 (local address 127.9.44.126:46205)
0808 23:27:22.192533 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:22.192540 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:22.193037 (+ 497us) server_negotiation.cc:407] Connection header received
0808 23:27:22.193263 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:22.193267 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:22.193344 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:22.193453 (+ 109us) server_negotiation.cc:226] Negotiated authn=JWT
0808 23:27:22.193987 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:22.194710 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:22.196726 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:22.197170 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:22.197264 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:22.197836 (+ 572us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 23:27:22.199953 (+ 2117us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41763/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 23:27:22.200300 (+ 347us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47842: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41763/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 23:27:22.200845 17422 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:22.192320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:46205 (local address 127.0.0.1:47842)
0808 23:27:22.192937 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:22.192955 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:22.193103 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:22.193494 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:22.193499 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:22.193512 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0808 23:27:22.193811 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:22.193823 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:22.194837 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:22.194843 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:22.196581 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:22.196597 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:22.197352 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:22.197358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:22.197560 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:22.197604 (+ 44us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 23:27:22.200181 (+ 2577us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41763/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 23:27:22.200426 (+ 245us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.44.126:46205: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41763/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":322,"threads_started":1}
W20260808 23:27:22.200898 17387 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47842: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41763/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260808 23:27:22.203531 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 17269
I20260808 23:27:22.204583 17374 generic_service.cc:196] Checking for leaks (request via RPC)
E20260808 23:27:22.212441 17424 webserver.cc:533] Webserver: sslize failed:
E20260808 23:27:22.212520 17424 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 23:27:22.293455 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 17269
2026-08-08T23:27:22Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (703 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T23:27:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:27:22Z Disabled control of system clock
I20260808 23:27:22.381541 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:33289
--webserver_interface=127.9.44.126
--webserver_port=0
--builtin_ntp_servers=127.9.44.84:39463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:33289
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786231597224934-9393-0/testchainca.cert with env {}
W20260808 23:27:22.690258 17432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:22.690550 17432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:22.690711 17432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:22.697538 17432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:27:22.697630 17432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:22.697675 17432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:22.697729 17432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:27:22.697772 17432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 23:27:22.697831 17432 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 23:27:22.697870 17432 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 23:27:22.707105 17432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:39463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:33289
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:33289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786231597224934-9393-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:22.708189 17432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:22.709357 17432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:22.720098 17440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:22.720054 17441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:22.720145 17449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:22.721153 17432 server_base.cc:1034] running on GCE node
I20260808 23:27:22.721855 17432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:22.723086 17432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:22.724277 17432 hybrid_clock.cc:648] HybridClock initialized: now 1786231642724232 us; error 33 us; skew 500 ppm
I20260808 23:27:22.724586 17432 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:27:22.726640 17432 webserver.cc:460] Webserver started at http://127.9.44.126:39609/ using document root <none> and password file <none>
I20260808 23:27:22.727279 17432 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:22.727383 17432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:22.727687 17432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:22.730331 17432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "41101a47dbec4ec19365d454b1c7e4e2"
format_stamp: "Formatted at 2026-08-08 23:27:22 on dist-test-slave-6bbx"
server_key: "62df380c1e4e760803732fb103d03596"
server_key_iv: "15e5115c7f0c350b0f543d00a9133897"
server_key_version: "encryptionkey@0"
I20260808 23:27:22.731066 17432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "41101a47dbec4ec19365d454b1c7e4e2"
format_stamp: "Formatted at 2026-08-08 23:27:22 on dist-test-slave-6bbx"
server_key: "62df380c1e4e760803732fb103d03596"
server_key_iv: "15e5115c7f0c350b0f543d00a9133897"
server_key_version: "encryptionkey@0"
I20260808 23:27:22.735752 17432 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260808 23:27:22.738870 17458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:22.740089 17432 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260808 23:27:22.740254 17432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "41101a47dbec4ec19365d454b1c7e4e2"
format_stamp: "Formatted at 2026-08-08 23:27:22 on dist-test-slave-6bbx"
server_key: "62df380c1e4e760803732fb103d03596"
server_key_iv: "15e5115c7f0c350b0f543d00a9133897"
server_key_version: "encryptionkey@0"
I20260808 23:27:22.740440 17432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:22.760555 17432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:22.761245 17432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:22.761490 17432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:22.778175 17432 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:33289
I20260808 23:27:22.778195 17536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:33289 every 8 connection(s)
I20260808 23:27:22.779333 17432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:27:22.782797 17539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:22.786737 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 17432
I20260808 23:27:22.786912 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:27:22.787513 9393 external_mini_cluster.cc:1428] Setting key 48f5122634645c222959059b29fa1fbc
I20260808 23:27:22.793314 17539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41101a47dbec4ec19365d454b1c7e4e2: Bootstrap starting.
I20260808 23:27:22.796705 17539 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41101a47dbec4ec19365d454b1c7e4e2: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:22.797879 17539 log.cc:824] T 00000000000000000000000000000000 P 41101a47dbec4ec19365d454b1c7e4e2: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:22.800621 17542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:22.789220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45842 (local address 127.9.44.126:33289)
0808 23:27:22.790145 (+ 925us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:22.790158 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:22.790199 (+ 41us) server_negotiation.cc:407] Connection header received
0808 23:27:22.791963 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:22.791986 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:22.792383 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:22.792669 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:22.793260 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:22.794533 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:22.796327 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:22.797008 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:22.797144 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:22.798304 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:22.798312 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:22.798329 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:27:22.799043 (+ 714us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:22.799136 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:27:22.799170 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:22.799276 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:22.799482 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 23:27:22.799770 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":311,"threads_started":1}
I20260808 23:27:22.800810 17539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41101a47dbec4ec19365d454b1c7e4e2: No bootstrap required, opened a new log
I20260808 23:27:22.803942 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:27:22.808354 17539 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41101a47dbec4ec19365d454b1c7e4e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41101a47dbec4ec19365d454b1c7e4e2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 33289 } }
I20260808 23:27:22.808663 17539 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41101a47dbec4ec19365d454b1c7e4e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:22.808765 17539 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41101a47dbec4ec19365d454b1c7e4e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41101a47dbec4ec19365d454b1c7e4e2, State: Initialized, Role: FOLLOWER
I20260808 23:27:22.809284 17539 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41101a47dbec4ec19365d454b1c7e4e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41101a47dbec4ec19365d454b1c7e4e2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 33289 } }
I20260808 23:27:22.809499 17539 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41101a47dbec4ec19365d454b1c7e4e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:27:22.809620 17539 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41101a47dbec4ec19365d454b1c7e4e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:27:22.809765 17539 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41101a47dbec4ec19365d454b1c7e4e2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:22.812109 17539 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41101a47dbec4ec19365d454b1c7e4e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41101a47dbec4ec19365d454b1c7e4e2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 33289 } }
I20260808 23:27:22.812562 17539 leader_election.cc:302] T 00000000000000000000000000000000 P 41101a47dbec4ec19365d454b1c7e4e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41101a47dbec4ec19365d454b1c7e4e2; no voters:
I20260808 23:27:22.813079 17539 leader_election.cc:288] T 00000000000000000000000000000000 P 41101a47dbec4ec19365d454b1c7e4e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:27:22.813198 17548 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41101a47dbec4ec19365d454b1c7e4e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:27:22.813822 17548 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41101a47dbec4ec19365d454b1c7e4e2 [term 1 LEADER]: Becoming Leader. State: Replica: 41101a47dbec4ec19365d454b1c7e4e2, State: Running, Role: LEADER
I20260808 23:27:22.814338 17548 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41101a47dbec4ec19365d454b1c7e4e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41101a47dbec4ec19365d454b1c7e4e2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 33289 } }
I20260808 23:27:22.814980 17539 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41101a47dbec4ec19365d454b1c7e4e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:27:22.816736 17550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41101a47dbec4ec19365d454b1c7e4e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41101a47dbec4ec19365d454b1c7e4e2. Latest consensus state: current_term: 1 leader_uuid: "41101a47dbec4ec19365d454b1c7e4e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41101a47dbec4ec19365d454b1c7e4e2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 33289 } } }
I20260808 23:27:22.817129 17550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41101a47dbec4ec19365d454b1c7e4e2 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:22.817633 17549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41101a47dbec4ec19365d454b1c7e4e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41101a47dbec4ec19365d454b1c7e4e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41101a47dbec4ec19365d454b1c7e4e2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 33289 } } }
I20260808 23:27:22.817966 17549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41101a47dbec4ec19365d454b1c7e4e2 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:22.819412 17556 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:27:22.824625 17556 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:27:22.837445 17556 catalog_manager.cc:1347] Generated new cluster ID: e52dcbc0b7824774bcaf4411242915f6
I20260808 23:27:22.837555 17556 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:27:22.856549 17556 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:27:22.858114 17556 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:27:22.863384 17556 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41101a47dbec4ec19365d454b1c7e4e2: Generated new TSK 0
I20260808 23:27:22.864187 17556 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:27:22.929008 17542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:22.877615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45858 (local address 127.9.44.126:33289)
0808 23:27:22.877785 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:22.877792 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:22.878159 (+ 367us) server_negotiation.cc:407] Connection header received
0808 23:27:22.878354 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:22.878359 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:22.878457 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:22.878629 (+ 172us) server_negotiation.cc:226] Negotiated authn=JWT
0808 23:27:22.879056 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:22.880162 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:22.881496 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:22.882170 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:22.882302 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:22.882731 (+ 429us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 23:27:22.928283 (+ 45552us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 23:27:22.928686 (+ 403us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45858: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 23:27:22.929143 17542 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45858: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 23:27:22.929131 17576 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:22.877444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:33289 (local address 127.0.0.1:45858)
0808 23:27:22.878071 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:22.878086 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:22.878199 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:22.878616 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:22.878619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:22.878628 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0808 23:27:22.878911 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:22.878919 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:22.880297 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:22.880301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:22.881350 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:22.881370 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:22.882315 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:22.882319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:22.882457 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:22.882486 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 23:27:22.928535 (+ 46049us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 23:27:22.928773 (+ 238us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.44.126:33289: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":365,"threads_started":1}
I20260808 23:27:22.930791 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 17432
I20260808 23:27:22.931921 17529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:23.025465 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 17432
2026-08-08T23:27:23Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (713 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (401 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (399 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:27:23 dist-test-slave-6bbx krb5kdc[17600](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:27:23 dist-test-slave-6bbx krb5kdc[17600](info): set up 2 sockets
Aug 08 23:27:23 dist-test-slave-6bbx krb5kdc[17600](info): commencing operation
krb5kdc: starting...
W20260808 23:27:25.002372 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:27:25 dist-test-slave-6bbx krb5kdc[17600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:27:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:27:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:27:25.212848 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:34075
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:41489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:34075
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:25.522300 17619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:25.522549 17619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:25.522636 17619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:25.529637 17619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:27:25.529736 17619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:25.529794 17619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:25.529850 17619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:27:25.529891 17619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:27:25.539160 17619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:41489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:34075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:34075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:25.540282 17619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:25.541450 17619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:25.552313 17629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:25.552450 17627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:25.552495 17635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:25.553210 17619 server_base.cc:1034] running on GCE node
I20260808 23:27:25.553912 17619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:25.555194 17619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:25.556452 17619 hybrid_clock.cc:648] HybridClock initialized: now 1786231645556396 us; error 32 us; skew 500 ppm
I20260808 23:27:25.556810 17619 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:25 dist-test-slave-6bbx krb5kdc[17600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231645, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:25.559754 17619 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:27:25.561070 17619 webserver.cc:460] Webserver started at http://127.9.44.126:45547/ using document root <none> and password file <none>
I20260808 23:27:25.561697 17619 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:25.561815 17619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:25.562114 17619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:25.564965 17619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "c391c6f37bf540c5b0ffe7e125777cea"
format_stamp: "Formatted at 2026-08-08 23:27:25 on dist-test-slave-6bbx"
server_key: "1cdd31d0335fe36d773b2e598339e36c"
server_key_iv: "af24ced9854deeca1ef2edc7e98c322b"
server_key_version: "encryptionkey@0"
I20260808 23:27:25.565704 17619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "c391c6f37bf540c5b0ffe7e125777cea"
format_stamp: "Formatted at 2026-08-08 23:27:25 on dist-test-slave-6bbx"
server_key: "1cdd31d0335fe36d773b2e598339e36c"
server_key_iv: "af24ced9854deeca1ef2edc7e98c322b"
server_key_version: "encryptionkey@0"
I20260808 23:27:25.570467 17619 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 23:27:25.573661 17644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:25.574903 17619 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 23:27:25.575059 17619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "c391c6f37bf540c5b0ffe7e125777cea"
format_stamp: "Formatted at 2026-08-08 23:27:25 on dist-test-slave-6bbx"
server_key: "1cdd31d0335fe36d773b2e598339e36c"
server_key_iv: "af24ced9854deeca1ef2edc7e98c322b"
server_key_version: "encryptionkey@0"
I20260808 23:27:25.575237 17619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:25.596325 17619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:25.600037 17619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:25.600347 17619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:25.617537 17619 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:34075
I20260808 23:27:25.617583 17732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:34075 every 8 connection(s)
I20260808 23:27:25.618764 17619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:27:25.622464 17734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:25.628178 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 17619
I20260808 23:27:25.628371 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:27:25.628996 9393 external_mini_cluster.cc:1428] Setting key 36f71bfa1975c9475d110473a913c946
I20260808 23:27:25.632510 17734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c391c6f37bf540c5b0ffe7e125777cea: Bootstrap starting.
I20260808 23:27:25.635813 17734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c391c6f37bf540c5b0ffe7e125777cea: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:25.636859 17734 log.cc:824] T 00000000000000000000000000000000 P c391c6f37bf540c5b0ffe7e125777cea: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:25.639545 17734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c391c6f37bf540c5b0ffe7e125777cea: No bootstrap required, opened a new log
Aug 08 23:27:25 dist-test-slave-6bbx krb5kdc[17600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:25.648113 17734 raft_consensus.cc:348] T 00000000000000000000000000000000 P c391c6f37bf540c5b0ffe7e125777cea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c391c6f37bf540c5b0ffe7e125777cea" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 34075 } }
I20260808 23:27:25.648484 17734 raft_consensus.cc:374] T 00000000000000000000000000000000 P c391c6f37bf540c5b0ffe7e125777cea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:25.648589 17734 raft_consensus.cc:729] T 00000000000000000000000000000000 P c391c6f37bf540c5b0ffe7e125777cea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c391c6f37bf540c5b0ffe7e125777cea, State: Initialized, Role: FOLLOWER
I20260808 23:27:25.649206 17734 consensus_queue.cc:260] T 00000000000000000000000000000000 P c391c6f37bf540c5b0ffe7e125777cea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c391c6f37bf540c5b0ffe7e125777cea" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 34075 } }
I20260808 23:27:25.649501 17734 raft_consensus.cc:388] T 00000000000000000000000000000000 P c391c6f37bf540c5b0ffe7e125777cea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:27:25.649662 17734 raft_consensus.cc:482] T 00000000000000000000000000000000 P c391c6f37bf540c5b0ffe7e125777cea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:27:25.649883 17734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c391c6f37bf540c5b0ffe7e125777cea [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:25.651295 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:25.630946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55492 (local address 127.9.44.126:34075)
0808 23:27:25.631729 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:25.631742 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:25.631783 (+ 41us) server_negotiation.cc:407] Connection header received
0808 23:27:25.633461 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:25.633483 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:25.633851 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:25.634167 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:25.635885 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:25.637252 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:25.638827 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:25.639462 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:25.639596 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:25.644234 (+ 4638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:25.644242 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:25.644255 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:25.644293 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:25.647271 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:25.648059 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:25.648066 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:25.648072 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:25.648254 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:25.648661 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:25.648666 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:25.648668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:25.649191 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:25.649455 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:25.649905 (+ 450us) server_negotiation.cc:299] Negotiation successful
0808 23:27:25.650284 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":325,"threads_started":1}
I20260808 23:27:25.652196 17734 raft_consensus.cc:504] T 00000000000000000000000000000000 P c391c6f37bf540c5b0ffe7e125777cea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c391c6f37bf540c5b0ffe7e125777cea" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 34075 } }
I20260808 23:27:25.652736 17734 leader_election.cc:302] T 00000000000000000000000000000000 P c391c6f37bf540c5b0ffe7e125777cea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c391c6f37bf540c5b0ffe7e125777cea; no voters:
I20260808 23:27:25.653347 17734 leader_election.cc:288] T 00000000000000000000000000000000 P c391c6f37bf540c5b0ffe7e125777cea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:27:25.653446 17740 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c391c6f37bf540c5b0ffe7e125777cea [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:27:25.654691 17740 raft_consensus.cc:686] T 00000000000000000000000000000000 P c391c6f37bf540c5b0ffe7e125777cea [term 1 LEADER]: Becoming Leader. State: Replica: c391c6f37bf540c5b0ffe7e125777cea, State: Running, Role: LEADER
I20260808 23:27:25.655200 17734 sys_catalog.cc:564] T 00000000000000000000000000000000 P c391c6f37bf540c5b0ffe7e125777cea [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:27:25.655145 17740 consensus_queue.cc:237] T 00000000000000000000000000000000 P c391c6f37bf540c5b0ffe7e125777cea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c391c6f37bf540c5b0ffe7e125777cea" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 34075 } }
I20260808 23:27:25.657588 17741 sys_catalog.cc:455] T 00000000000000000000000000000000 P c391c6f37bf540c5b0ffe7e125777cea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c391c6f37bf540c5b0ffe7e125777cea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c391c6f37bf540c5b0ffe7e125777cea" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 34075 } } }
I20260808 23:27:25.657963 17741 sys_catalog.cc:458] T 00000000000000000000000000000000 P c391c6f37bf540c5b0ffe7e125777cea [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:25.658504 17742 sys_catalog.cc:455] T 00000000000000000000000000000000 P c391c6f37bf540c5b0ffe7e125777cea [sys.catalog]: SysCatalogTable state changed. Reason: New leader c391c6f37bf540c5b0ffe7e125777cea. Latest consensus state: current_term: 1 leader_uuid: "c391c6f37bf540c5b0ffe7e125777cea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c391c6f37bf540c5b0ffe7e125777cea" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 34075 } } }
I20260808 23:27:25.658818 17742 sys_catalog.cc:458] T 00000000000000000000000000000000 P c391c6f37bf540c5b0ffe7e125777cea [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:25.658936 17750 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:27:25.665617 17750 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:27:25.675145 17750 catalog_manager.cc:1347] Generated new cluster ID: 69c6a784bd5c432c893f4f5bd70ce545
I20260808 23:27:25.675316 17750 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:27:25.703034 17750 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:27:25.704922 17750 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:27:25.719893 17750 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c391c6f37bf540c5b0ffe7e125777cea: Generated new TSK 0
I20260808 23:27:25.720713 17750 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.44.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
I20260808 23:27:25.779412 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:0
--local_ip_for_outbound_sockets=127.9.44.65
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:34075
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:41489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:26.112159 17768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:26.112432 17768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:26.112538 17768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:26.119647 17768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:26.119779 17768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:26.119901 17768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:27:26.129549 17768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:41489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:34075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:26.130738 17768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:26.132035 17768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:26.144806 17773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:26.144730 17779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:26.144778 17777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:26.145717 17768 server_base.cc:1034] running on GCE node
I20260808 23:27:26.146296 17768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:26.147169 17768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:26.148394 17768 hybrid_clock.cc:648] HybridClock initialized: now 1786231646148325 us; error 44 us; skew 500 ppm
I20260808 23:27:26.148700 17768 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:26 dist-test-slave-6bbx krb5kdc[17600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231646, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:26.154366 17768 init.cc:377] Logged in from keytab as kudu/127.9.44.65@KRBTEST.COM (short username kudu)
I20260808 23:27:26.155880 17768 webserver.cc:460] Webserver started at http://127.9.44.65:43589/ using document root <none> and password file <none>
I20260808 23:27:26.156528 17768 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:26.156632 17768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:26.156936 17768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:26.160169 17768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-0/data/instance:
uuid: "02041ee164cb462aa9a5b79f81524528"
format_stamp: "Formatted at 2026-08-08 23:27:26 on dist-test-slave-6bbx"
server_key: "4d70f062586f7e5990b0d2d3bb8a8e1b"
server_key_iv: "6a0c18e63c951ae411156875ca6cc863"
server_key_version: "encryptionkey@0"
I20260808 23:27:26.160938 17768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance:
uuid: "02041ee164cb462aa9a5b79f81524528"
format_stamp: "Formatted at 2026-08-08 23:27:26 on dist-test-slave-6bbx"
server_key: "4d70f062586f7e5990b0d2d3bb8a8e1b"
server_key_iv: "6a0c18e63c951ae411156875ca6cc863"
server_key_version: "encryptionkey@0"
I20260808 23:27:26.165943 17768 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 23:27:26.169348 17788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:26.170565 17768 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 23:27:26.170704 17768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "02041ee164cb462aa9a5b79f81524528"
format_stamp: "Formatted at 2026-08-08 23:27:26 on dist-test-slave-6bbx"
server_key: "4d70f062586f7e5990b0d2d3bb8a8e1b"
server_key_iv: "6a0c18e63c951ae411156875ca6cc863"
server_key_version: "encryptionkey@0"
I20260808 23:27:26.170903 17768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:26.201076 17768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:26.205111 17768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:26.205369 17768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:26.206316 17768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:26.207692 17768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:26.207772 17768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:26.207886 17768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:26.207944 17768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:26.238124 17768 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:45021
I20260808 23:27:26.238170 17956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:45021 every 8 connection(s)
I20260808 23:27:26.239380 17768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:27:26.242447 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 17768
I20260808 23:27:26.242656 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance
I20260808 23:27:26.243286 9393 external_mini_cluster.cc:1428] Setting key 675ada4872455473ba9af8f991a0a431
Aug 08 23:27:26 dist-test-slave-6bbx krb5kdc[17600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231646, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:26.259548 17959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:26.242061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:45385 (local address 127.9.44.126:34075)
0808 23:27:26.242584 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:26.242590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:26.243648 (+ 1058us) server_negotiation.cc:407] Connection header received
0808 23:27:26.245691 (+ 2043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:26.245697 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:26.245778 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:26.245907 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:26.247333 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:26.248310 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:26.249823 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:26.250381 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:26.250508 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:26.253739 (+ 3231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:26.253746 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:26.253750 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:26.253787 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:26.256445 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:26.257300 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:26.257305 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:26.257307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:26.257450 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:26.257919 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:26.257922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:26.257924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:26.258185 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:26.258296 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:26.259096 (+ 800us) server_negotiation.cc:299] Negotiation successful
0808 23:27:26.259270 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":271,"threads_started":1}
I20260808 23:27:26.260998 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:26.242620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:34075 (local address 127.9.44.65:45385)
0808 23:27:26.243465 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:26.243516 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:26.245432 (+ 1916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:26.246074 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:26.246082 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:26.246605 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:26.247157 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:26.247174 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:26.248452 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:26.248456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:26.249684 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:26.249696 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:26.250562 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:26.250567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:26.250757 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:26.251362 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:26.251385 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:26.253532 (+ 2147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:26.256588 (+ 3056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:26.256596 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:26.256611 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:26.257158 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:26.257571 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:26.257577 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:26.257580 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:26.257827 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:26.258306 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:26.258314 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:26.258711 (+ 397us) client_negotiation.cc:769] Sending connection context
0808 23:27:26.259033 (+ 322us) client_negotiation.cc:240] Negotiation successful
0808 23:27:26.259562 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":355,"threads_started":1}
I20260808 23:27:26.262569 17957 heartbeater.cc:347] Connected to a master server at 127.9.44.126:34075
I20260808 23:27:26.262895 17957 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:26.263782 17957 heartbeater.cc:511] Master 127.9.44.126:34075 requested a full tablet report, sending...
I20260808 23:27:26.265996 17673 ts_manager.cc:194] Registered new tserver with Master: 02041ee164cb462aa9a5b79f81524528 (127.9.44.65:45021)
I20260808 23:27:26.267439 17673 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.65@KRBTEST.COM'} at 127.9.44.65:45385
WARNING: no policy specified for kudu/127.9.44.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
I20260808 23:27:26.295612 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:0
--local_ip_for_outbound_sockets=127.9.44.66
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:34075
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:41489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:26.611814 17965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:26.612094 17965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:26.612207 17965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:26.620245 17965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:26.620362 17965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:26.620537 17965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:27:26.635079 17965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:41489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:34075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:26.636425 17965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:26.637939 17965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:26.650415 17973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:26.650347 17975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:26.650381 17977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:26.651248 17965 server_base.cc:1034] running on GCE node
I20260808 23:27:26.651873 17965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:26.652724 17965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:26.653925 17965 hybrid_clock.cc:648] HybridClock initialized: now 1786231646653877 us; error 34 us; skew 500 ppm
I20260808 23:27:26.654211 17965 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:26 dist-test-slave-6bbx krb5kdc[17600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231646, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:26.657256 17965 init.cc:377] Logged in from keytab as kudu/127.9.44.66@KRBTEST.COM (short username kudu)
I20260808 23:27:26.658595 17965 webserver.cc:460] Webserver started at http://127.9.44.66:38965/ using document root <none> and password file <none>
I20260808 23:27:26.659215 17965 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:26.659332 17965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:26.659626 17965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:26.662413 17965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-1/data/instance:
uuid: "4414d09319e14318b87f41a8ff54a023"
format_stamp: "Formatted at 2026-08-08 23:27:26 on dist-test-slave-6bbx"
server_key: "9d7c2773b202a8e9bd29c9049ff2fcb2"
server_key_iv: "39a4f6d9141066a56fa03c7c53908efe"
server_key_version: "encryptionkey@0"
I20260808 23:27:26.663134 17965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance:
uuid: "4414d09319e14318b87f41a8ff54a023"
format_stamp: "Formatted at 2026-08-08 23:27:26 on dist-test-slave-6bbx"
server_key: "9d7c2773b202a8e9bd29c9049ff2fcb2"
server_key_iv: "39a4f6d9141066a56fa03c7c53908efe"
server_key_version: "encryptionkey@0"
I20260808 23:27:26.667781 17965 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260808 23:27:26.670900 17987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:26.672125 17965 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 23:27:26.672272 17965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "4414d09319e14318b87f41a8ff54a023"
format_stamp: "Formatted at 2026-08-08 23:27:26 on dist-test-slave-6bbx"
server_key: "9d7c2773b202a8e9bd29c9049ff2fcb2"
server_key_iv: "39a4f6d9141066a56fa03c7c53908efe"
server_key_version: "encryptionkey@0"
I20260808 23:27:26.672441 17965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:26.684564 17965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:26.687963 17965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:26.688207 17965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:26.689028 17965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:26.690402 17965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:26.690490 17965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:26.690606 17965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:26.690670 17965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:26.720608 17965 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:33109
I20260808 23:27:26.720672 18157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:33109 every 8 connection(s)
I20260808 23:27:26.721791 17965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:27:26.723698 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 17965
I20260808 23:27:26.723875 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance
I20260808 23:27:26.724448 9393 external_mini_cluster.cc:1428] Setting key b7560d59982882c39703e32eb5d8d698
Aug 08 23:27:26 dist-test-slave-6bbx krb5kdc[17600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231646, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:26.743499 17959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:26.724331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:38833 (local address 127.9.44.126:34075)
0808 23:27:26.724496 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:26.724501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:26.726002 (+ 1501us) server_negotiation.cc:407] Connection header received
0808 23:27:26.728349 (+ 2347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:26.728353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:26.728427 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:26.728506 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:26.730363 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:26.731246 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:26.733362 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:26.733885 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:26.733987 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:26.737815 (+ 3828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:26.737820 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:26.737823 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:26.737855 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:26.740477 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:26.741312 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:26.741316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:26.741318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:26.741440 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:26.741888 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:26.741891 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:26.741893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:26.742182 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:26.742290 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:26.743082 (+ 792us) server_negotiation.cc:299] Negotiation successful
0808 23:27:26.743264 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 23:27:26.744930 18160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:26.724952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:34075 (local address 127.9.44.66:38833)
0808 23:27:26.725827 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:26.725887 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:26.728058 (+ 2171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:26.728769 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:26.728779 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:26.729444 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:26.730188 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:26.730211 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:26.731431 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:26.731436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:26.733230 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:26.733245 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:26.734097 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:26.734104 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:26.734362 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:26.735051 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:26.735077 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:26.737599 (+ 2522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:26.740614 (+ 3015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:26.740622 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:26.740636 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:26.741197 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:26.741559 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:26.741563 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:26.741566 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:26.741793 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:26.742304 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:26.742311 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:26.742686 (+ 375us) client_negotiation.cc:769] Sending connection context
0808 23:27:26.743027 (+ 341us) client_negotiation.cc:240] Negotiation successful
0808 23:27:26.743513 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":376,"threads_started":1}
I20260808 23:27:26.746239 18158 heartbeater.cc:347] Connected to a master server at 127.9.44.126:34075
I20260808 23:27:26.746526 18158 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:26.747288 18158 heartbeater.cc:511] Master 127.9.44.126:34075 requested a full tablet report, sending...
I20260808 23:27:26.748746 17673 ts_manager.cc:194] Registered new tserver with Master: 4414d09319e14318b87f41a8ff54a023 (127.9.44.66:33109)
I20260808 23:27:26.749562 17673 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.66@KRBTEST.COM'} at 127.9.44.66:38833
WARNING: no policy specified for kudu/127.9.44.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
I20260808 23:27:26.775730 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:0
--local_ip_for_outbound_sockets=127.9.44.67
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:34075
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:41489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:27.092542 18168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:27.092821 18168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:27.092957 18168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:27.100122 18168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:27.100234 18168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:27.100369 18168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:27:27.109509 18168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:41489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:34075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:27.110658 18168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:27.111905 18168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:27.124382 18173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:27.124346 18175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:27.124423 18179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:27.125067 18168 server_base.cc:1034] running on GCE node
I20260808 23:27:27.125659 18168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:27.126514 18168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:27.127735 18168 hybrid_clock.cc:648] HybridClock initialized: now 1786231647127676 us; error 44 us; skew 500 ppm
I20260808 23:27:27.128017 18168 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:27 dist-test-slave-6bbx krb5kdc[17600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231647, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:27.131148 18168 init.cc:377] Logged in from keytab as kudu/127.9.44.67@KRBTEST.COM (short username kudu)
I20260808 23:27:27.132507 18168 webserver.cc:460] Webserver started at http://127.9.44.67:39277/ using document root <none> and password file <none>
I20260808 23:27:27.133112 18168 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:27.133217 18168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:27.133471 18168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:27.136234 18168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-2/data/instance:
uuid: "a140f70f6e0f437da8ce09812a3cb316"
format_stamp: "Formatted at 2026-08-08 23:27:27 on dist-test-slave-6bbx"
server_key: "78fab96f964ff9d1f2679f97630ad238"
server_key_iv: "54345f94bf5d2f657f45d516f174a4bf"
server_key_version: "encryptionkey@0"
I20260808 23:27:27.136950 18168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance:
uuid: "a140f70f6e0f437da8ce09812a3cb316"
format_stamp: "Formatted at 2026-08-08 23:27:27 on dist-test-slave-6bbx"
server_key: "78fab96f964ff9d1f2679f97630ad238"
server_key_iv: "54345f94bf5d2f657f45d516f174a4bf"
server_key_version: "encryptionkey@0"
I20260808 23:27:27.141707 18168 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 23:27:27.144881 18189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:27.146034 18168 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 23:27:27.146188 18168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "a140f70f6e0f437da8ce09812a3cb316"
format_stamp: "Formatted at 2026-08-08 23:27:27 on dist-test-slave-6bbx"
server_key: "78fab96f964ff9d1f2679f97630ad238"
server_key_iv: "54345f94bf5d2f657f45d516f174a4bf"
server_key_version: "encryptionkey@0"
I20260808 23:27:27.146366 18168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:27.165212 18168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:27.168982 18168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:27.169248 18168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:27.170089 18168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:27.171309 18168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:27.171388 18168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:27.171509 18168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:27.171569 18168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:27.201648 18168 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:34797
I20260808 23:27:27.201709 18364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:34797 every 8 connection(s)
I20260808 23:27:27.202802 18168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:27:27.212194 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 18168
I20260808 23:27:27.212354 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance
I20260808 23:27:27.212765 9393 external_mini_cluster.cc:1428] Setting key 52d09345bc65d3fbd84db5bd4920f812
Aug 08 23:27:27 dist-test-slave-6bbx krb5kdc[17600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231647, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:27.221118 17959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:27.205182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:52965 (local address 127.9.44.126:34075)
0808 23:27:27.205360 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:27.205365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:27.206427 (+ 1062us) server_negotiation.cc:407] Connection header received
0808 23:27:27.208201 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:27.208216 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:27.208298 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:27.208387 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:27.209780 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:27.210534 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:27.212092 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:27.212558 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:27.212674 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:27.216035 (+ 3361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:27.216041 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:27.216044 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:27.216076 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:27.218343 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:27.218993 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:27.218998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:27.219000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:27.219126 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:27.219519 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:27.219522 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:27.219525 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:27.219815 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:27.219945 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:27.220581 (+ 636us) server_negotiation.cc:299] Negotiation successful
0808 23:27:27.220792 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 23:27:27.221887 18367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:27.205584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:34075 (local address 127.9.44.67:52965)
0808 23:27:27.206284 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:27.206319 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:27.207933 (+ 1614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:27.208618 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:27.208626 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:27.209097 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:27.209611 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:27.209628 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:27.210692 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:27.210695 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:27.211959 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:27.211973 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:27.212790 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:27.212799 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:27.213021 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:27.213602 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:27.213641 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:27.215845 (+ 2204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:27.218466 (+ 2621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:27.218475 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:27.218489 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:27.218874 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:27.219227 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:27.219232 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:27.219236 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:27.219408 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:27.219931 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:27.219938 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:27.220224 (+ 286us) client_negotiation.cc:769] Sending connection context
0808 23:27:27.220444 (+ 220us) client_negotiation.cc:240] Negotiation successful
0808 23:27:27.220910 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":278,"threads_started":1}
I20260808 23:27:27.223011 18365 heartbeater.cc:347] Connected to a master server at 127.9.44.126:34075
I20260808 23:27:27.223273 18365 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:27.223963 18365 heartbeater.cc:511] Master 127.9.44.126:34075 requested a full tablet report, sending...
I20260808 23:27:27.225185 17673 ts_manager.cc:194] Registered new tserver with Master: a140f70f6e0f437da8ce09812a3cb316 (127.9.44.67:34797)
I20260808 23:27:27.225960 17673 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.67@KRBTEST.COM'} at 127.9.44.67:52965
I20260808 23:27:27.227533 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/test-admin.keytab.
Aug 08 23:27:27 dist-test-slave-6bbx krb5kdc[17600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:27.258468 9393 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 23:27:27.263267 18374 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:27.261044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40154 (local address 127.9.44.65:45021)
0808 23:27:27.261659 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:27.261666 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:27.261693 (+ 27us) server_negotiation.cc:407] Connection header received
0808 23:27:27.261753 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:27.261758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:27.261958 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:27.262097 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:27.262846 (+ 749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40154: BlockingRecv error: recv got EOF from 127.0.0.1:40154 (error 108)
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":305,"threads_started":1}
W20260808 23:27:27.263404 9393 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/krb5cc' of type FILE
Aug 08 23:27:27 dist-test-slave-6bbx krb5kdc[17600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:27.264889 9393 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 23:27:27 dist-test-slave-6bbx krb5kdc[17600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:27.266108 9393 init.cc:295] Successfully reacquired a new kerberos TGT
I20260808 23:27:27.270411 17957 heartbeater.cc:503] Master 127.9.44.126:34075 was elected leader, sending a full tablet report...
Aug 08 23:27:27 dist-test-slave-6bbx krb5kdc[17600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.65@KRBTEST.COM
I20260808 23:27:27.280575 18374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:27.267545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40160 (local address 127.9.44.65:45021)
0808 23:27:27.267716 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:27.267720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:27.267995 (+ 275us) server_negotiation.cc:407] Connection header received
0808 23:27:27.268108 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:27.268112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:27.268164 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:27.268241 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:27.269104 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:27.270052 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:27.271398 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:27.272188 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:27.272284 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:27.275481 (+ 3197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:27.275491 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:27.275504 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:27.275534 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:27.278263 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:27.278825 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:27.278831 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:27.278833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:27.278946 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:27.279254 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:27.279258 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:27.279260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:27.279590 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:27.279853 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:27.280118 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 23:27:27.280308 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 23:27:27.285470 18374 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:27.284013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40166 (local address 127.9.44.65:45021)
0808 23:27:27.284190 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:27.284196 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:27.284378 (+ 182us) server_negotiation.cc:407] Connection header received
0808 23:27:27.284487 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:27.284490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:27.284539 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:27.284609 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:27.285269 (+ 660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40166: BlockingRecv error: recv got EOF from 127.0.0.1:40166 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 23:27:27.285873 9393 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/krb5cc' of type FILE
Aug 08 23:27:27 dist-test-slave-6bbx krb5kdc[17600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:27.287098 9393 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 23:27:27 dist-test-slave-6bbx krb5kdc[17600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:27.288272 9393 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 23:27:27 dist-test-slave-6bbx krb5kdc[17600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.65@KRBTEST.COM
I20260808 23:27:27.300004 18374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:27.289318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40172 (local address 127.9.44.65:45021)
0808 23:27:27.289436 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:27.289440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:27.289742 (+ 302us) server_negotiation.cc:407] Connection header received
0808 23:27:27.289840 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:27.289843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:27.289888 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:27.289956 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:27.290766 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:27.291384 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:27.292479 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:27.292867 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:27.292950 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:27.295989 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:27.295993 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:27.295996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:27.296022 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:27.298067 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:27.298653 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:27.298656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:27.298658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:27.298780 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:27.299050 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:27.299053 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:27.299055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:27.299273 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:27.299451 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:27.299642 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 23:27:27.299814 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 23:27:27.303344 18374 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:27.302060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40176 (local address 127.9.44.65:45021)
0808 23:27:27.302244 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:27.302249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:27.302363 (+ 114us) server_negotiation.cc:407] Connection header received
0808 23:27:27.302455 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:27.302458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:27.302520 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:27.302589 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:27.303144 (+ 555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40176: BlockingRecv error: recv got EOF from 127.0.0.1:40176 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260808 23:27:27.303481 9393 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786231597224934-9393-0/krb5kdc/krb5cc' of type FILE
Aug 08 23:27:27 dist-test-slave-6bbx krb5kdc[17600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:27.304824 9393 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 23:27:27 dist-test-slave-6bbx krb5kdc[17600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:27.305928 9393 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 23:27:27 dist-test-slave-6bbx krb5kdc[17600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.65@KRBTEST.COM
I20260808 23:27:27.318070 18374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:27.307046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40190 (local address 127.9.44.65:45021)
0808 23:27:27.307226 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:27.307232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:27.307334 (+ 102us) server_negotiation.cc:407] Connection header received
0808 23:27:27.307462 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:27.307467 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:27.307523 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:27.307596 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:27.308486 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:27.309128 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:27.310310 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:27.310716 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:27.310818 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:27.313753 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:27.313758 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:27.313761 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:27.313787 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:27.315882 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:27.316432 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:27.316437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:27.316440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:27.316575 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:27.316914 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:27.316918 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:27.316920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:27.317152 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:27.317455 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:27.317638 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 23:27:27.317823 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 23:27:27.319429 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 17768
I20260808 23:27:27.329437 18374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:27.320010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40196 (local address 127.9.44.65:45021)
0808 23:27:27.320140 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:27.320143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:27.320435 (+ 292us) server_negotiation.cc:407] Connection header received
0808 23:27:27.320536 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:27.320538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:27.320583 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:27.320650 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:27.321483 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:27.322088 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:27.323177 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:27.323560 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:27.323641 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:27.325351 (+ 1710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:27.325356 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:27.325359 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:27.325383 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:27.327548 (+ 2165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:27.328104 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:27.328108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:27.328110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:27.328229 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:27.328479 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:27.328482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:27.328484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:27.328687 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:27.328875 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:27.329076 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 23:27:27.329216 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 23:27:27.329919 17951 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:27.410395 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 17768
I20260808 23:27:27.428953 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 17965
Aug 08 23:27:27 dist-test-slave-6bbx krb5kdc[17600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.66@KRBTEST.COM
I20260808 23:27:27.443384 18397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:27.429757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57314 (local address 127.9.44.66:33109)
0808 23:27:27.430302 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:27.430309 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:27.430335 (+ 26us) server_negotiation.cc:407] Connection header received
0808 23:27:27.430399 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:27.430405 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:27.430612 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:27.430762 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:27.431816 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:27.432826 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:27.434076 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:27.434633 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:27.434760 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:27.438460 (+ 3700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:27.438471 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:27.438485 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:27.438518 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:27.441059 (+ 2541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:27.441644 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:27.441650 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:27.441652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:27.441766 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:27.442054 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:27.442057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:27.442059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:27.442394 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:27.442627 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:27.442840 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 23:27:27.443073 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":248,"threads_started":1}
I20260808 23:27:27.444630 18152 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:27.524709 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 17965
I20260808 23:27:27.545751 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 18168
Aug 08 23:27:27 dist-test-slave-6bbx krb5kdc[17600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.67@KRBTEST.COM
I20260808 23:27:27.560570 18399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:27.546654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51684 (local address 127.9.44.67:34797)
0808 23:27:27.547320 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:27.547327 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:27.547365 (+ 38us) server_negotiation.cc:407] Connection header received
0808 23:27:27.547429 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:27.547436 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:27.547652 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:27.547820 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:27.548760 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:27.549826 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:27.551134 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:27.551870 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:27.551970 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:27.555541 (+ 3571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:27.555552 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:27.555585 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:27.555622 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:27.558201 (+ 2579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:27.558777 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:27.558782 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:27.558785 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:27.558909 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:27.559185 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:27.559187 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:27.559189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:27.559514 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:27.559793 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:27.560015 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 23:27:27.560260 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":356,"threads_started":1}
I20260808 23:27:27.561892 18358 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:27.641938 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 18168
I20260808 23:27:27.660172 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 17619
I20260808 23:27:27.661352 17728 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:27.760664 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 17619
2026-08-08T23:27:27Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3937 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:27:27 dist-test-slave-6bbx krb5kdc[18406](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:27:27 dist-test-slave-6bbx krb5kdc[18406](info): set up 2 sockets
Aug 08 23:27:27 dist-test-slave-6bbx krb5kdc[18406](info): commencing operation
krb5kdc: starting...
W20260808 23:27:28.951365 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:27:29 dist-test-slave-6bbx krb5kdc[18406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:27:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:27:29Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.126.keytab.
Entry for principal oryx/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.126.keytab.
I20260808 23:27:29.157090 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:37605
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.126.keytab
--principal=oryx/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:40499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:37605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:29.468340 18428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:29.468600 18428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:29.468695 18428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:29.475560 18428 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:27:29.475656 18428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:29.475744 18428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:29.475805 18428 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:27:29.475850 18428 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:27:29.485301 18428 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:40499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:37605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.126.keytab
--principal=oryx/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:37605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:29.486449 18428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:29.487703 18428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:29.498593 18438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:29.498647 18433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:29.498672 18435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:29.499365 18428 server_base.cc:1034] running on GCE node
I20260808 23:27:29.500133 18428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:29.501446 18428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:29.502646 18428 hybrid_clock.cc:648] HybridClock initialized: now 1786231649502601 us; error 30 us; skew 500 ppm
I20260808 23:27:29.502980 18428 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:29 dist-test-slave-6bbx krb5kdc[18406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231649, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:29.506043 18428 init.cc:377] Logged in from keytab as oryx/127.9.44.126@KRBTEST.COM (short username oryx)
I20260808 23:27:29.507409 18428 webserver.cc:460] Webserver started at http://127.9.44.126:43755/ using document root <none> and password file <none>
I20260808 23:27:29.508061 18428 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:29.508173 18428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:29.508431 18428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:29.511204 18428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "e1653d349ee64235a944e5c763fc4945"
format_stamp: "Formatted at 2026-08-08 23:27:29 on dist-test-slave-6bbx"
server_key: "b5e05f126496bc41c26d923c4dd0f2f4"
server_key_iv: "b08d1bd8e889cae9ddbc91087f6f35c0"
server_key_version: "encryptionkey@0"
I20260808 23:27:29.511965 18428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "e1653d349ee64235a944e5c763fc4945"
format_stamp: "Formatted at 2026-08-08 23:27:29 on dist-test-slave-6bbx"
server_key: "b5e05f126496bc41c26d923c4dd0f2f4"
server_key_iv: "b08d1bd8e889cae9ddbc91087f6f35c0"
server_key_version: "encryptionkey@0"
I20260808 23:27:29.516835 18428 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 23:27:29.520005 18447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:29.521196 18428 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260808 23:27:29.521327 18428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "e1653d349ee64235a944e5c763fc4945"
format_stamp: "Formatted at 2026-08-08 23:27:29 on dist-test-slave-6bbx"
server_key: "b5e05f126496bc41c26d923c4dd0f2f4"
server_key_iv: "b08d1bd8e889cae9ddbc91087f6f35c0"
server_key_version: "encryptionkey@0"
I20260808 23:27:29.521497 18428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:29.559895 18428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:29.563539 18428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:29.563802 18428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:29.580957 18428 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:37605
I20260808 23:27:29.580991 18525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:37605 every 8 connection(s)
I20260808 23:27:29.582250 18428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:27:29.582638 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 18428
I20260808 23:27:29.582829 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:27:29.583714 9393 external_mini_cluster.cc:1428] Setting key 9fca75384ebc966be847b81667fad8de
I20260808 23:27:29.586614 18527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 23:27:29 dist-test-slave-6bbx krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM
I20260808 23:27:29.599218 18527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945: Bootstrap starting.
I20260808 23:27:29.602272 18527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:29.603459 18527 log.cc:824] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:29.605875 18530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:29.585766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41948 (local address 127.9.44.126:37605)
0808 23:27:29.586530 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:29.586540 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:29.586581 (+ 41us) server_negotiation.cc:407] Connection header received
0808 23:27:29.588242 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:29.588267 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:29.588616 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:29.588949 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:29.590666 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:29.592046 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:29.593744 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:29.594371 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:29.594490 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:29.599020 (+ 4530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:29.599028 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:29.599040 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:29.599078 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:29.601825 (+ 2747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:29.602655 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:29.602662 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:29.602668 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:29.602831 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:29.603218 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:29.603222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:29.603224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:29.603795 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:29.604105 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:29.604531 (+ 426us) server_negotiation.cc:299] Negotiation successful
0808 23:27:29.604908 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":347,"threads_started":1}
I20260808 23:27:29.606272 18527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945: No bootstrap required, opened a new log
I20260808 23:27:29.613131 18527 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } }
I20260808 23:27:29.613443 18527 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:29.613535 18527 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1653d349ee64235a944e5c763fc4945, State: Initialized, Role: FOLLOWER
I20260808 23:27:29.614077 18527 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } }
I20260808 23:27:29.614301 18527 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:27:29.614434 18527 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:27:29.614581 18527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:29.616611 18527 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } }
I20260808 23:27:29.617120 18527 leader_election.cc:302] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1653d349ee64235a944e5c763fc4945; no voters:
I20260808 23:27:29.617743 18527 leader_election.cc:288] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:27:29.617812 18533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:27:29.618502 18533 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [term 1 LEADER]: Becoming Leader. State: Replica: e1653d349ee64235a944e5c763fc4945, State: Running, Role: LEADER
I20260808 23:27:29.619019 18533 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } }
I20260808 23:27:29.619695 18527 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:27:29.621692 18536 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1653d349ee64235a944e5c763fc4945. Latest consensus state: current_term: 1 leader_uuid: "e1653d349ee64235a944e5c763fc4945" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } }
I20260808 23:27:29.622043 18536 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:29.622925 18534 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1653d349ee64235a944e5c763fc4945" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } }
I20260808 23:27:29.623345 18534 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:29.626015 18542 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:27:29.631050 18542 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:27:29.639134 18542 catalog_manager.cc:1347] Generated new cluster ID: 072e9d2097594b749e6fc0670c92e369
I20260808 23:27:29.639245 18542 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:27:29.658417 18542 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:27:29.659584 18542 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:27:29.667425 18542 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945: Generated new TSK 0
I20260808 23:27:29.668357 18542 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.44.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.44.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.44.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.65.keytab.
Entry for principal oryx/127.9.44.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.65.keytab.
I20260808 23:27:29.731423 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:0
--local_ip_for_outbound_sockets=127.9.44.65
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:37605
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.65.keytab
--principal=oryx/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:40499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:30.053920 18564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:30.054183 18564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:30.054325 18564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:30.062639 18564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:30.062749 18564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:30.062908 18564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:27:30.072191 18564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:40499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.65.keytab
--principal=oryx/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:37605
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:30.073323 18564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:30.074618 18564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:30.086946 18571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:30.087020 18575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:30.087059 18570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:30.087986 18564 server_base.cc:1034] running on GCE node
I20260808 23:27:30.088551 18564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:30.089385 18564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:30.090586 18564 hybrid_clock.cc:648] HybridClock initialized: now 1786231650090530 us; error 45 us; skew 500 ppm
I20260808 23:27:30.090869 18564 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:30 dist-test-slave-6bbx krb5kdc[18406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231650, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:30.093806 18564 init.cc:377] Logged in from keytab as oryx/127.9.44.65@KRBTEST.COM (short username oryx)
I20260808 23:27:30.095153 18564 webserver.cc:460] Webserver started at http://127.9.44.65:35237/ using document root <none> and password file <none>
I20260808 23:27:30.095824 18564 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:30.095942 18564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:30.096217 18564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:30.098940 18564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/data/instance:
uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2"
format_stamp: "Formatted at 2026-08-08 23:27:30 on dist-test-slave-6bbx"
server_key: "0def10741324d574f44131c463dbce38"
server_key_iv: "b01c5a8cabacee80d2fd032d0ac18839"
server_key_version: "encryptionkey@0"
I20260808 23:27:30.099689 18564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance:
uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2"
format_stamp: "Formatted at 2026-08-08 23:27:30 on dist-test-slave-6bbx"
server_key: "0def10741324d574f44131c463dbce38"
server_key_iv: "b01c5a8cabacee80d2fd032d0ac18839"
server_key_version: "encryptionkey@0"
I20260808 23:27:30.104470 18564 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 23:27:30.107788 18588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:30.108974 18564 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 23:27:30.109117 18564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2"
format_stamp: "Formatted at 2026-08-08 23:27:30 on dist-test-slave-6bbx"
server_key: "0def10741324d574f44131c463dbce38"
server_key_iv: "b01c5a8cabacee80d2fd032d0ac18839"
server_key_version: "encryptionkey@0"
I20260808 23:27:30.109303 18564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:30.127241 18564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:30.130788 18564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:30.131034 18564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:30.131955 18564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:30.133183 18564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:30.133260 18564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:30.133387 18564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:30.133440 18564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:30.162897 18564 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:42143
I20260808 23:27:30.162956 18777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:42143 every 8 connection(s)
I20260808 23:27:30.164089 18564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:27:30.173095 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 18564
I20260808 23:27:30.173300 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance
I20260808 23:27:30.174108 9393 external_mini_cluster.cc:1428] Setting key 27c53a5e390eff5ede6b1bee49f1e412
Aug 08 23:27:30 dist-test-slave-6bbx krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231650, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM
I20260808 23:27:30.183619 18780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:30.166416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:41045 (local address 127.9.44.126:37605)
0808 23:27:30.166831 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:30.166837 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:30.167840 (+ 1003us) server_negotiation.cc:407] Connection header received
0808 23:27:30.169622 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:30.169629 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:30.169707 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:30.169823 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:30.171289 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:30.172045 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:30.174140 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:30.174553 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:30.174665 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:30.178369 (+ 3704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:30.178374 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:30.178377 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:30.178408 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:30.180809 (+ 2401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:30.181463 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:30.181469 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:30.181471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:30.181590 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:30.181997 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:30.182002 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:30.182005 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:30.182285 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:30.182417 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:30.183044 (+ 627us) server_negotiation.cc:299] Negotiation successful
0808 23:27:30.183259 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":215,"threads_started":1}
I20260808 23:27:30.184468 18785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:30.166950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.9.44.65:41045)
0808 23:27:30.167686 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:30.167739 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:30.169354 (+ 1615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:30.170077 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:30.170085 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:30.170587 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:30.171117 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:30.171135 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:30.172209 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:30.172213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:30.173988 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:30.174005 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:30.174766 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:30.174771 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:30.175008 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:30.175698 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:30.175732 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:30.178163 (+ 2431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:30.180936 (+ 2773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:30.180944 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:30.180958 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:30.181351 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:30.181691 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:30.181696 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:30.181700 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:30.181888 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:30.182409 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:30.182416 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:30.182702 (+ 286us) client_negotiation.cc:769] Sending connection context
0808 23:27:30.182940 (+ 238us) client_negotiation.cc:240] Negotiation successful
0808 23:27:30.183404 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":267,"threads_started":1}
I20260808 23:27:30.185896 18778 heartbeater.cc:347] Connected to a master server at 127.9.44.126:37605
I20260808 23:27:30.186182 18778 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:30.186990 18778 heartbeater.cc:511] Master 127.9.44.126:37605 requested a full tablet report, sending...
I20260808 23:27:30.189317 18466 ts_manager.cc:194] Registered new tserver with Master: f1ac6dffcc5c43dcaaec28cb6b1130f2 (127.9.44.65:42143)
I20260808 23:27:30.191236 18466 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.44.65@KRBTEST.COM'} at 127.9.44.65:41045
WARNING: no policy specified for oryx/127.9.44.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.44.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.44.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.66.keytab.
Entry for principal oryx/127.9.44.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.66.keytab.
I20260808 23:27:30.224263 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:0
--local_ip_for_outbound_sockets=127.9.44.66
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:37605
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.66.keytab
--principal=oryx/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:40499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:30.537803 18792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:30.538069 18792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:30.538214 18792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:30.545115 18792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:30.545222 18792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:30.545380 18792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:27:30.554492 18792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:40499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.66.keytab
--principal=oryx/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:37605
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:30.555637 18792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:30.556908 18792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:30.569298 18804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:30.569284 18801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:30.569736 18799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:30.569970 18792 server_base.cc:1034] running on GCE node
I20260808 23:27:30.570587 18792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:30.571491 18792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:30.572687 18792 hybrid_clock.cc:648] HybridClock initialized: now 1786231650572631 us; error 44 us; skew 500 ppm
I20260808 23:27:30.572991 18792 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:30 dist-test-slave-6bbx krb5kdc[18406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231650, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:30.575985 18792 init.cc:377] Logged in from keytab as oryx/127.9.44.66@KRBTEST.COM (short username oryx)
I20260808 23:27:30.577368 18792 webserver.cc:460] Webserver started at http://127.9.44.66:42901/ using document root <none> and password file <none>
I20260808 23:27:30.578027 18792 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:30.578148 18792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:30.578446 18792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:30.581400 18792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/data/instance:
uuid: "a140e60268f342a29eff122f628bbab5"
format_stamp: "Formatted at 2026-08-08 23:27:30 on dist-test-slave-6bbx"
server_key: "967a45a35b06ef25ddfbf97c3fa03abc"
server_key_iv: "f002cb6a5ff17c4c95441ae4ee67bca9"
server_key_version: "encryptionkey@0"
I20260808 23:27:30.582149 18792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance:
uuid: "a140e60268f342a29eff122f628bbab5"
format_stamp: "Formatted at 2026-08-08 23:27:30 on dist-test-slave-6bbx"
server_key: "967a45a35b06ef25ddfbf97c3fa03abc"
server_key_iv: "f002cb6a5ff17c4c95441ae4ee67bca9"
server_key_version: "encryptionkey@0"
I20260808 23:27:30.587261 18792 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 23:27:30.590744 18813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:30.592141 18792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:27:30.592283 18792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "a140e60268f342a29eff122f628bbab5"
format_stamp: "Formatted at 2026-08-08 23:27:30 on dist-test-slave-6bbx"
server_key: "967a45a35b06ef25ddfbf97c3fa03abc"
server_key_iv: "f002cb6a5ff17c4c95441ae4ee67bca9"
server_key_version: "encryptionkey@0"
I20260808 23:27:30.592463 18792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:30.620873 18792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:30.624596 18792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:30.624895 18792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:30.625789 18792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:30.627082 18792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:30.627159 18792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:30.627277 18792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:30.627336 18792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:30.656733 18792 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:39169
I20260808 23:27:30.656767 18996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:39169 every 8 connection(s)
I20260808 23:27:30.657930 18792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:27:30.662264 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 18792
I20260808 23:27:30.662427 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance
I20260808 23:27:30.662865 9393 external_mini_cluster.cc:1428] Setting key bc506f89712cc50ff7d1d356158a1096
Aug 08 23:27:30 dist-test-slave-6bbx krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231650, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM
I20260808 23:27:30.676995 18780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:30.660367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:58975 (local address 127.9.44.126:37605)
0808 23:27:30.660533 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:30.660538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:30.661610 (+ 1072us) server_negotiation.cc:407] Connection header received
0808 23:27:30.663530 (+ 1920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:30.663534 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:30.663622 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:30.663756 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:30.665493 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:30.666218 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:30.667745 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:30.668154 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:30.668274 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:30.671458 (+ 3184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:30.671463 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:30.671466 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:30.671496 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:30.673884 (+ 2388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:30.674731 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:30.674736 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:30.674738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:30.674876 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:30.675311 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:30.675314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:30.675316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:30.675579 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:30.675754 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:30.676415 (+ 661us) server_negotiation.cc:299] Negotiation successful
0808 23:27:30.676661 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 23:27:30.677917 19000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:30.660776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.9.44.66:58975)
0808 23:27:30.661457 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:30.661499 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:30.663222 (+ 1723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:30.663965 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:30.663976 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:30.664659 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:30.665331 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:30.665349 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:30.666363 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:30.666367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:30.667579 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:30.667592 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:30.668366 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:30.668374 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:30.668589 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:30.669184 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:30.669206 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:30.671258 (+ 2052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:30.674009 (+ 2751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:30.674019 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:30.674034 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:30.674591 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:30.674997 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:30.675003 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:30.675007 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:30.675196 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:30.675761 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:30.675769 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:30.676071 (+ 302us) client_negotiation.cc:769] Sending connection context
0808 23:27:30.676313 (+ 242us) client_negotiation.cc:240] Negotiation successful
0808 23:27:30.676803 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":273,"threads_started":1}
I20260808 23:27:30.679304 18997 heartbeater.cc:347] Connected to a master server at 127.9.44.126:37605
I20260808 23:27:30.679622 18997 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:30.680531 18997 heartbeater.cc:511] Master 127.9.44.126:37605 requested a full tablet report, sending...
I20260808 23:27:30.682155 18466 ts_manager.cc:194] Registered new tserver with Master: a140e60268f342a29eff122f628bbab5 (127.9.44.66:39169)
I20260808 23:27:30.683243 18466 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.44.66@KRBTEST.COM'} at 127.9.44.66:58975
WARNING: no policy specified for oryx/127.9.44.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.44.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.44.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.67.keytab.
Entry for principal oryx/127.9.44.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.67.keytab.
I20260808 23:27:30.712430 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:0
--local_ip_for_outbound_sockets=127.9.44.67
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:37605
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.67.keytab
--principal=oryx/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:40499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:31.074177 19008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:31.074712 19008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:31.074892 19008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:31.087423 19008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:31.087563 19008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:31.087821 19008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:27:31.101572 19008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:40499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.67.keytab
--principal=oryx/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:37605
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:31.103502 19008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:31.105561 19008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:31.124284 19020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:31.124197 19015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:31.124912 19024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:31.125809 19008 server_base.cc:1034] running on GCE node
I20260808 23:27:31.126602 19008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:31.127923 19008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:31.129168 19008 hybrid_clock.cc:648] HybridClock initialized: now 1786231651129052 us; error 92 us; skew 500 ppm
I20260808 23:27:31.129582 19008 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:31 dist-test-slave-6bbx krb5kdc[18406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231651, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:31.133905 19008 init.cc:377] Logged in from keytab as oryx/127.9.44.67@KRBTEST.COM (short username oryx)
I20260808 23:27:31.135917 19008 webserver.cc:460] Webserver started at http://127.9.44.67:35655/ using document root <none> and password file <none>
I20260808 23:27:31.136865 19008 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:31.137002 19008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:31.137425 19008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:31.141490 19008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/data/instance:
uuid: "518525b775c4450a8105feaa7126331b"
format_stamp: "Formatted at 2026-08-08 23:27:31 on dist-test-slave-6bbx"
server_key: "2a88675930efce1df8e5585717b6b18e"
server_key_iv: "1a21e4198870fca4a7b4ff1fdc818344"
server_key_version: "encryptionkey@0"
I20260808 23:27:31.142969 19008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance:
uuid: "518525b775c4450a8105feaa7126331b"
format_stamp: "Formatted at 2026-08-08 23:27:31 on dist-test-slave-6bbx"
server_key: "2a88675930efce1df8e5585717b6b18e"
server_key_iv: "1a21e4198870fca4a7b4ff1fdc818344"
server_key_version: "encryptionkey@0"
I20260808 23:27:31.152784 19008 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.008s sys 0.000s
I20260808 23:27:31.158825 19036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:31.161037 19008 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 23:27:31.161309 19008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "518525b775c4450a8105feaa7126331b"
format_stamp: "Formatted at 2026-08-08 23:27:31 on dist-test-slave-6bbx"
server_key: "2a88675930efce1df8e5585717b6b18e"
server_key_iv: "1a21e4198870fca4a7b4ff1fdc818344"
server_key_version: "encryptionkey@0"
I20260808 23:27:31.161674 19008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:31.193820 19008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:31.194618 18778 heartbeater.cc:503] Master 127.9.44.126:37605 was elected leader, sending a full tablet report...
I20260808 23:27:31.201916 19008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:31.202273 19008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:31.203708 19008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:31.205876 19008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:31.206025 19008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:31.206228 19008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:31.206318 19008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:31.249941 19008 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:33679
I20260808 23:27:31.250012 19208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:33679 every 8 connection(s)
I20260808 23:27:31.251623 19008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:27:31.253676 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 19008
I20260808 23:27:31.253868 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance
I20260808 23:27:31.254487 9393 external_mini_cluster.cc:1428] Setting key 00a24d731ac5e437d2cf727d3d9c9ba4
Aug 08 23:27:31 dist-test-slave-6bbx krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231651, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM
I20260808 23:27:31.280419 19212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:31.257398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:45819 (local address 127.9.44.126:37605)
0808 23:27:31.258252 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:31.258262 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:31.259913 (+ 1651us) server_negotiation.cc:407] Connection header received
0808 23:27:31.262467 (+ 2554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:31.262476 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:31.262562 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:31.262677 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:31.264710 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:31.265771 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:31.267938 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:31.268508 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:31.268627 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:31.273259 (+ 4632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:31.273267 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:31.273270 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:31.273315 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:31.276756 (+ 3441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:31.277644 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:31.277649 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:31.277651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:31.277845 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:31.278377 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:31.278380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:31.278386 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:31.278762 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:31.278922 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:31.279766 (+ 844us) server_negotiation.cc:299] Negotiation successful
0808 23:27:31.279983 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":408,"threads_started":1}
I20260808 23:27:31.281613 19215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:31.258438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.9.44.67:45819)
0808 23:27:31.259630 (+ 1192us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:31.259745 (+ 115us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:31.262138 (+ 2393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:31.262975 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:31.262987 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:31.263741 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:31.264520 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:31.264550 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:31.265916 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:31.265919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:31.267791 (+ 1872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:31.267808 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:31.268737 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:31.268752 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:31.269042 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:31.269932 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:31.269959 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:31.272996 (+ 3037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:31.276909 (+ 3913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:31.276917 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:31.276940 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:31.277502 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:31.277969 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:31.277974 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:31.277976 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:31.278258 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:31.278890 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:31.278898 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:31.279289 (+ 391us) client_negotiation.cc:769] Sending connection context
0808 23:27:31.279594 (+ 305us) client_negotiation.cc:240] Negotiation successful
0808 23:27:31.280185 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":771,"thread_start_us":485,"threads_started":1}
I20260808 23:27:31.283572 19209 heartbeater.cc:347] Connected to a master server at 127.9.44.126:37605
I20260808 23:27:31.284089 19209 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:31.285063 19209 heartbeater.cc:511] Master 127.9.44.126:37605 requested a full tablet report, sending...
I20260808 23:27:31.286804 18464 ts_manager.cc:194] Registered new tserver with Master: 518525b775c4450a8105feaa7126331b (127.9.44.67:33679)
I20260808 23:27:31.288060 18464 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.44.67@KRBTEST.COM'} at 127.9.44.67:45819
I20260808 23:27:31.297168 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.44.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.44.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.44.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.125.keytab.
Entry for principal oryx/127.9.44.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.125.keytab.
I20260808 23:27:31.361956 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.125:39361
--webserver_interface=127.9.44.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.125.keytab
--principal=oryx/127.9.44.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:40499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:37605,127.9.44.125:39361
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:27:31.686303 18997 heartbeater.cc:503] Master 127.9.44.126:37605 was elected leader, sending a full tablet report...
W20260808 23:27:31.829069 19218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:31.829504 19218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:31.829720 19218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:31.839747 19218 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:27:31.839885 19218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:31.839982 19218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:31.840078 19218 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:27:31.840195 19218 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:27:31.853667 19218 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:40499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:37605,127.9.44.125:39361
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.125.keytab
--principal=oryx/127.9.44.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.125:39361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.44.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
W20260808 23:27:31.854146 19218 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.9.44.126:37605,127.9.44.125:39361'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 23:27:31.855527 19218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:31.857368 19218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:31.872560 19223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:31.872586 19224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:31.872659 19227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:31.874338 19218 server_base.cc:1034] running on GCE node
I20260808 23:27:31.875444 19218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:31.877467 19218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:31.878758 19218 hybrid_clock.cc:648] HybridClock initialized: now 1786231651878673 us; error 47 us; skew 500 ppm
I20260808 23:27:31.879256 19218 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:31 dist-test-slave-6bbx krb5kdc[18406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231651, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:31.883546 19218 init.cc:377] Logged in from keytab as oryx/127.9.44.125@KRBTEST.COM (short username oryx)
I20260808 23:27:31.885476 19218 webserver.cc:460] Webserver started at http://127.9.44.125:43601/ using document root <none> and password file <none>
I20260808 23:27:31.886435 19218 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:31.886571 19218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:31.887030 19218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:31.891181 19218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-1/data/instance:
uuid: "d6758c943e0b4c7b89118814fd429e93"
format_stamp: "Formatted at 2026-08-08 23:27:31 on dist-test-slave-6bbx"
server_key: "26d5d528b17326a15697c945c2c9926a"
server_key_iv: "4248dced4470c9be361afac9734c0cdc"
server_key_version: "encryptionkey@0"
I20260808 23:27:31.892297 19218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-1/wal/instance:
uuid: "d6758c943e0b4c7b89118814fd429e93"
format_stamp: "Formatted at 2026-08-08 23:27:31 on dist-test-slave-6bbx"
server_key: "26d5d528b17326a15697c945c2c9926a"
server_key_iv: "4248dced4470c9be361afac9734c0cdc"
server_key_version: "encryptionkey@0"
I20260808 23:27:31.899718 19218 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260808 23:27:31.904490 19240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:31.906433 19218 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260808 23:27:31.906666 19218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-1/wal
uuid: "d6758c943e0b4c7b89118814fd429e93"
format_stamp: "Formatted at 2026-08-08 23:27:31 on dist-test-slave-6bbx"
server_key: "26d5d528b17326a15697c945c2c9926a"
server_key_iv: "4248dced4470c9be361afac9734c0cdc"
server_key_version: "encryptionkey@0"
I20260808 23:27:31.906942 19218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:31.961594 19218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:31.966948 19218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:31.967250 19218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:31.991241 19218 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.125:39361
I20260808 23:27:31.991262 19316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.125:39361 every 8 connection(s)
I20260808 23:27:31.993108 19218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-1/data/info.pb
I20260808 23:27:31.998451 19317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:31.998740 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 19218
I20260808 23:27:32.012840 19317 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:27:32 dist-test-slave-6bbx krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.44.125@KRBTEST.COM
Aug 08 23:27:32 dist-test-slave-6bbx krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231651, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.125@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM
I20260808 23:27:32.033514 19322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:32.001864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55204 (local address 127.9.44.125:39361)
0808 23:27:32.002971 (+ 1107us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:32.002994 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:32.003055 (+ 61us) server_negotiation.cc:407] Connection header received
0808 23:27:32.005601 (+ 2546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:32.005649 (+ 48us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:32.006198 (+ 549us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:32.006643 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:32.009347 (+ 2704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.011187 (+ 1840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.013899 (+ 2712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.014876 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.015047 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:32.022009 (+ 6962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:32.022022 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:32.022047 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:32.022098 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:32.027173 (+ 5075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:32.028378 (+ 1205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:32.028387 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:32.028396 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:32.028637 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:32.029282 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:32.029292 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:32.029295 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:32.030368 (+ 1073us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:32.030823 (+ 455us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:32.031362 (+ 539us) server_negotiation.cc:299] Negotiation successful
0808 23:27:32.031908 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":804,"thread_start_us":509,"threads_started":1}
I20260808 23:27:32.038933 19325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:32.016800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.0.0.1:41958)
0808 23:27:32.018011 (+ 1211us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:32.018067 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:32.018284 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:32.018846 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:32.018852 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:32.019909 (+ 1057us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:32.020401 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:32.020415 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.022302 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.022313 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:32.024934 (+ 2621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:32.024954 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.026088 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.026095 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:32.026336 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:32.027112 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:32.027146 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:32.031132 (+ 3986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:32.035635 (+ 4503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:32.035642 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:32.035646 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:32.036286 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:32.036782 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:32.036786 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:32.036799 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:32.036978 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:32.037874 (+ 896us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:32.037881 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:32.038085 (+ 204us) client_negotiation.cc:769] Sending connection context
0808 23:27:32.038320 (+ 235us) client_negotiation.cc:240] Negotiation successful
0808 23:27:32.038603 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":861,"thread_start_us":675,"threads_started":1}
I20260808 23:27:32.038939 19324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:32.016800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41958 (local address 127.9.44.126:37605)
0808 23:27:32.017478 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:32.017489 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:32.018216 (+ 727us) server_negotiation.cc:407] Connection header received
0808 23:27:32.018462 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:32.018471 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:32.018611 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:32.018786 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:32.020615 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.022135 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.025104 (+ 2969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.025822 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.026037 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:32.031413 (+ 5376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:32.031421 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:32.031438 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:32.031487 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:32.035471 (+ 3984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:32.036453 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:32.036458 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:32.036461 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:32.036645 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:32.037153 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:32.037157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:32.037159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:32.037688 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:32.037878 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:32.038440 (+ 562us) server_negotiation.cc:299] Negotiation successful
0808 23:27:32.038650 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":334,"threads_started":1}
I20260808 23:27:32.041093 19317 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:27:32 dist-test-slave-6bbx krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231651, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.125@KRBTEST.COM for oryx/127.9.44.125@KRBTEST.COM
I20260808 23:27:32.056325 19325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:32.041783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:39361 (local address 127.0.0.1:55216)
0808 23:27:32.042014 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:32.042063 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:32.042219 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:32.042608 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:32.042612 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:32.043067 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:32.043222 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:32.043231 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.044464 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.044471 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:32.045577 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:32.045592 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.046541 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.046549 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:32.046814 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:32.047377 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:32.047392 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:32.049884 (+ 2492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:32.053343 (+ 3459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:32.053359 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:32.053362 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:32.053910 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:32.054299 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:32.054304 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:32.054311 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:32.054443 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:32.055033 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:32.055038 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:32.055241 (+ 203us) client_negotiation.cc:769] Sending connection context
0808 23:27:32.055407 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 23:27:32.055790 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 23:27:32.056591 19322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:32.041892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55216 (local address 127.9.44.125:39361)
0808 23:27:32.042132 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:32.042137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:32.042172 (+ 35us) server_negotiation.cc:407] Connection header received
0808 23:27:32.042340 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:32.042346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:32.042442 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:32.042554 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:32.043377 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.044319 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.045744 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.046298 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.046513 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:32.050079 (+ 3566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:32.050088 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:32.050092 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:32.050125 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:32.053173 (+ 3048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:32.054030 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:32.054040 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:32.054043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:32.054174 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:32.054570 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:32.054575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:32.054586 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:32.054896 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:32.055102 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:32.055535 (+ 433us) server_negotiation.cc:299] Negotiation successful
0808 23:27:32.055998 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":166,"server-negotiator.queue_time_us":67}
I20260808 23:27:32.062356 19317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93: Bootstrap starting.
I20260808 23:27:32.067618 19317 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:32.069365 19317 log.cc:824] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:32.072991 19317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93: No bootstrap required, opened a new log
I20260808 23:27:32.082636 19317 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } }
I20260808 23:27:32.083068 19317 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:32.083199 19317 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6758c943e0b4c7b89118814fd429e93, State: Initialized, Role: FOLLOWER
I20260808 23:27:32.084163 19317 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } }
I20260808 23:27:32.085368 19327 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } } }
I20260808 23:27:32.085773 19327 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:27:32.086728 19317 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 23:27:32.098384 19340 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:27:32.099139 19340 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:27:32.103025 19218 master_runner.cc:428] Detected that this master d6758c943e0b4c7b89118814fd429e93 is joining an existing cluster
I20260808 23:27:32.103178 19218 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.44.125:39361
I20260808 23:27:32.131829 19324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:32.109028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41960 (local address 127.9.44.126:37605)
0808 23:27:32.109304 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:32.109324 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:32.110060 (+ 736us) server_negotiation.cc:407] Connection header received
0808 23:27:32.110622 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:32.110628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:32.110788 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:32.110928 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:32.113648 (+ 2720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.115074 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.117511 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.118601 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.118819 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:32.122538 (+ 3719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:32.122546 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:32.122555 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:32.122622 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:32.128008 (+ 5386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:32.129007 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:32.129014 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:32.129027 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:32.129215 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:32.129769 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:32.129775 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:32.129779 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:32.130166 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:32.130330 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:32.130898 (+ 568us) server_negotiation.cc:299] Negotiation successful
0808 23:27:32.131194 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 23:27:32.132067 19348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:32.109027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.0.0.1:41960)
0808 23:27:32.109945 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:32.109968 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:32.110332 (+ 364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:32.111271 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:32.111281 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:32.112556 (+ 1275us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:32.113183 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:32.113215 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.115276 (+ 2061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.115281 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:32.117301 (+ 2020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:32.117324 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.118884 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.118896 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:32.119294 (+ 398us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:32.120357 (+ 1063us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:32.120428 (+ 71us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:32.122240 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:32.128162 (+ 5922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:32.128167 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:32.128170 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:32.128828 (+ 658us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:32.129350 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:32.129354 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:32.129357 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:32.129611 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:32.130313 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:32.130318 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:32.130527 (+ 209us) client_negotiation.cc:769] Sending connection context
0808 23:27:32.130706 (+ 179us) client_negotiation.cc:240] Negotiation successful
0808 23:27:32.131194 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"mutex_wait_us":103,"thread_start_us":503,"threads_started":1}
I20260808 23:27:32.132184 19322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:32.110504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55224 (local address 127.9.44.125:39361)
0808 23:27:32.110871 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:32.110881 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:32.111865 (+ 984us) server_negotiation.cc:407] Connection header received
0808 23:27:32.112288 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:32.112293 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:32.112443 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:32.112590 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:32.114666 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.116370 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.118878 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.120030 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.120147 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:32.123370 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:32.123376 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:32.123380 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:32.123435 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:32.127366 (+ 3931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:32.128290 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:32.128297 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:32.128304 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:32.128524 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:32.129299 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:32.129304 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:32.129308 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:32.129936 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:32.130187 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:32.131015 (+ 828us) server_negotiation.cc:299] Negotiation successful
0808 23:27:32.131418 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":114,"server-negotiator.queue_time_us":115}
I20260808 23:27:32.133347 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:32.110053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:39361 (local address 127.0.0.1:55224)
0808 23:27:32.111740 (+ 1687us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:32.111762 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:32.112065 (+ 303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:32.112612 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:32.112616 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:32.113931 (+ 1315us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:32.114439 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:32.114463 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.116671 (+ 2208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.116681 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:32.118644 (+ 1963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:32.118660 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.120696 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.120704 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:32.120961 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:32.121852 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:32.121887 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:32.123140 (+ 1253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:32.127527 (+ 4387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:32.127532 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:32.127536 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:32.128097 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:32.128739 (+ 642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:32.128749 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:32.128752 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:32.129059 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:32.130284 (+ 1225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:32.130289 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:32.130486 (+ 197us) client_negotiation.cc:769] Sending connection context
0808 23:27:32.130755 (+ 269us) client_negotiation.cc:240] Negotiation successful
0808 23:27:32.132692 (+ 1937us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1490,"mutex_wait_us":1463,"thread_start_us":1236,"threads_started":1}
Aug 08 23:27:32 dist-test-slave-6bbx krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231649, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.126@KRBTEST.COM for oryx/127.9.44.125@KRBTEST.COM
I20260808 23:27:32.168350 19350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:32.141701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:39361 (local address 127.0.0.1:55232)
0808 23:27:32.142615 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:32.142713 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:32.142933 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:32.146571 (+ 3638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:32.146583 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:32.147825 (+ 1242us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:32.148248 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:32.148265 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.150260 (+ 1995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.150265 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:32.152354 (+ 2089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:32.152413 (+ 59us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.153854 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.153872 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:32.154154 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:32.155048 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:32.155085 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:32.159034 (+ 3949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:32.163819 (+ 4785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:32.163832 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:32.163851 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:32.164725 (+ 874us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:32.165490 (+ 765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:32.165495 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:32.165499 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:32.165829 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:32.166948 (+ 1119us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:32.166959 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:32.167254 (+ 295us) client_negotiation.cc:769] Sending connection context
0808 23:27:32.167546 (+ 292us) client_negotiation.cc:240] Negotiation successful
0808 23:27:32.167929 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":352,"threads_started":1}
I20260808 23:27:32.169553 19322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:32.143883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55232 (local address 127.9.44.125:39361)
0808 23:27:32.145891 (+ 2008us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:32.145908 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:32.145926 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:27:32.146071 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:32.146089 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:32.146277 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:32.146472 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:32.148469 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.150091 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.152594 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.153610 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.153829 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:32.159276 (+ 5447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:32.159281 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:32.159292 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:32.159336 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:32.163569 (+ 4233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:32.165011 (+ 1442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:32.165025 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:32.165032 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:32.165298 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:32.166025 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:32.166035 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:32.166038 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:32.166589 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:32.166777 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:32.168798 (+ 2021us) server_negotiation.cc:299] Negotiation successful
0808 23:27:32.169078 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1864}
I20260808 23:27:32.172143 18464 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.44.125:39361: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d6758c943e0b4c7b89118814fd429e93"
member_type: NON_VOTER
last_known_addr {
host: "127.9.44.125"
port: 39361
}
attrs {
promote: true
}
}
dest_uuid: "e1653d349ee64235a944e5c763fc4945"
cas_config_opid_index: -1
I20260808 23:27:32.174086 18464 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: NON_VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: true } }
WARNING: no policy specified for oryx/127.9.44.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.44.124@KRBTEST.COM" created.
I20260808 23:27:32.183408 19353 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER d6758c943e0b4c7b89118814fd429e93 (127.9.44.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: NON_VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: true } } }
I20260808 23:27:32.184979 19279 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:32.190753 19353 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.44.125:39361
I20260808 23:27:32.190809 19279 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [term 1 FOLLOWER]: Refusing update from remote peer e1653d349ee64235a944e5c763fc4945: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 23:27:32.192153 19218 master.cc:537] Master@127.9.44.125:39361 shutting down...
I20260808 23:27:32.194200 19353 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: NON_VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:27:32.194756 19355 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e1653d349ee64235a944e5c763fc4945" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: NON_VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: true } } }
I20260808 23:27:32.195235 19355 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: This master's current role is: LEADER
W20260808 23:27:32.209092 18451 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 23:27:32.211376 18451 consensus_peers.cc:487] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 -> Peer d6758c943e0b4c7b89118814fd429e93 (127.9.44.125:39361): Couldn't send request to peer d6758c943e0b4c7b89118814fd429e93. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 23:27:32.213971 19218 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:27:32.214452 19218 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:27:32.214732 19218 tablet_replica.cc:330] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93: stopping tablet replica
I20260808 23:27:32.222481 19218 master.cc:552] Master@127.9.44.125:39361 shutdown complete.
I20260808 23:27:32.222671 19218 master_runner.cc:305] Clearing existing system tablet
I20260808 23:27:32.223346 19218 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:32.223471 19218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:32.227089 19218 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260808 23:27:32.228976 19365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:32.229357 19218 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:32.229718 19218 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 23:27:32.229848 19218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-1/wal
uuid: "d6758c943e0b4c7b89118814fd429e93"
format_stamp: "Formatted at 2026-08-08 23:27:31 on dist-test-slave-6bbx"
server_key: "26d5d528b17326a15697c945c2c9926a"
server_key_iv: "4248dced4470c9be361afac9734c0cdc"
server_key_version: "encryptionkey@0"
I20260808 23:27:32.236133 19218 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93: Deleting tablet data with delete state TABLET_DATA_DELETED
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.44.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.124.keytab.
Entry for principal oryx/127.9.44.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.124.keytab.
I20260808 23:27:32.240231 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.124:41697
--webserver_interface=127.9.44.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.124.keytab
--principal=oryx/127.9.44.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:40499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:37605,127.9.44.125:39361,127.9.44.124:41697
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:27:32.242952 19218 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 23:27:32.243131 19218 log.cc:1196] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 23:27:32.243542 19218 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93: Deleting consensus metadata
I20260808 23:27:32.243940 19218 master_runner.cc:315] Copying system tablet from 127.9.44.126:37605
I20260808 23:27:32.260365 19218 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93: tablet copy: Beginning tablet copy session from remote peer at address 127.9.44.126:37605
I20260808 23:27:32.277829 19324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:32.261671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41968 (local address 127.9.44.126:37605)
0808 23:27:32.261956 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:32.261964 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:32.262631 (+ 667us) server_negotiation.cc:407] Connection header received
0808 23:27:32.262919 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:32.262935 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:32.263051 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:32.263200 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:32.264473 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.265528 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.267227 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.267996 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.268170 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:32.270693 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:32.270698 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:32.270701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:32.270753 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:32.274245 (+ 3492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:32.275015 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:32.275021 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:32.275028 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:32.275239 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:32.275812 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:32.275837 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:32.275842 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:32.276249 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:32.276439 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:32.276998 (+ 559us) server_negotiation.cc:299] Negotiation successful
0808 23:27:32.277287 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260808 23:27:32.277987 19379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:32.261623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.0.0.1:41968)
0808 23:27:32.262486 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:32.262505 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:32.262712 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:32.263227 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:32.263232 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:32.263934 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:32.264275 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:32.264297 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.265722 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.265726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:32.267002 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:32.267016 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.268180 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.268191 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:32.268452 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:32.269266 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:32.269295 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:32.270389 (+ 1094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:32.274387 (+ 3998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:32.274396 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:32.274400 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:32.274874 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:32.275381 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:32.275386 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:32.275394 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:32.275599 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:32.276393 (+ 794us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:32.276398 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:32.276621 (+ 223us) client_negotiation.cc:769] Sending connection context
0808 23:27:32.276825 (+ 204us) client_negotiation.cc:240] Negotiation successful
0808 23:27:32.277237 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":517,"threads_started":1}
I20260808 23:27:32.279385 18499 tablet_copy_service.cc:140] P e1653d349ee64235a944e5c763fc4945: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d6758c943e0b4c7b89118814fd429e93 ({username='oryx', principal='oryx/127.9.44.125@KRBTEST.COM'} at 127.0.0.1:41968)
I20260808 23:27:32.279934 18499 tablet_copy_service.cc:161] P e1653d349ee64235a944e5c763fc4945: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d6758c943e0b4c7b89118814fd429e93 at {username='oryx', principal='oryx/127.9.44.125@KRBTEST.COM'} at 127.0.0.1:41968: session id = d6758c943e0b4c7b89118814fd429e93-00000000000000000000000000000000
I20260808 23:27:32.284723 18499 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 23:27:32.288147 19218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 23:27:32.291401 19209 heartbeater.cc:503] Master 127.9.44.126:37605 was elected leader, sending a full tablet report...
I20260808 23:27:32.296211 19218 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93: tablet copy: Starting download of 0 data blocks...
I20260808 23:27:32.296691 19218 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93: tablet copy: Starting download of 1 WAL segments...
I20260808 23:27:32.300756 19218 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 23:27:32.305087 18499 tablet_copy_service.cc:342] P e1653d349ee64235a944e5c763fc4945: Request end of tablet copy session d6758c943e0b4c7b89118814fd429e93-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.44.125@KRBTEST.COM'} at 127.0.0.1:41968
I20260808 23:27:32.305387 18499 tablet_copy_service.cc:433] P e1653d349ee64235a944e5c763fc4945: ending tablet copy session d6758c943e0b4c7b89118814fd429e93-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d6758c943e0b4c7b89118814fd429e93
I20260808 23:27:32.322232 19218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:32.325877 19381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:32.326606 19382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:32.335306 19384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:32.335768 19218 server_base.cc:1034] running on GCE node
I20260808 23:27:32.336179 19218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:32.337177 19218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:32.338472 19218 hybrid_clock.cc:648] HybridClock initialized: now 1786231652338384 us; error 29 us; skew 500 ppm
I20260808 23:27:32.338970 19218 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:32 dist-test-slave-6bbx krb5kdc[18406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231652, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:32.342157 19218 init.cc:377] Logged in from keytab as oryx/127.9.44.125@KRBTEST.COM (short username oryx)
I20260808 23:27:32.343945 19218 webserver.cc:460] Webserver started at http://127.9.44.125:35855/ using document root <none> and password file <none>
I20260808 23:27:32.344511 19218 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:32.344673 19218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:32.349116 19218 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260808 23:27:32.352037 19394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:32.352684 19218 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260808 23:27:32.352943 19218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-1/wal
uuid: "d6758c943e0b4c7b89118814fd429e93"
format_stamp: "Formatted at 2026-08-08 23:27:31 on dist-test-slave-6bbx"
server_key: "26d5d528b17326a15697c945c2c9926a"
server_key_iv: "4248dced4470c9be361afac9734c0cdc"
server_key_version: "encryptionkey@0"
I20260808 23:27:32.353181 19218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:32.383108 19218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:32.389140 19218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:32.429080 19218 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.125:39361
I20260808 23:27:32.429116 19480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.125:39361 every 8 connection(s)
I20260808 23:27:32.430603 19218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-1/data/info.pb
I20260808 23:27:32.439893 19481 sys_catalog.cc:263] Verifying existing consensus state
I20260808 23:27:32.445371 19481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93: Bootstrap starting.
I20260808 23:27:32.473018 19481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:27:32.473812 19481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93: Bootstrap complete.
I20260808 23:27:32.475404 19481 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: NON_VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: true } }
I20260808 23:27:32.475878 19481 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d6758c943e0b4c7b89118814fd429e93, State: Initialized, Role: LEARNER
I20260808 23:27:32.476300 19481 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: NON_VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: true } }
I20260808 23:27:32.477551 19484 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: NON_VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: true } } }
I20260808 23:27:32.478044 19481 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:27:32.478194 19484 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [sys.catalog]: This master's current role is: LEARNER
I20260808 23:27:32.488834 19508 catalog_manager.cc:1259] Loaded cluster ID: 072e9d2097594b749e6fc0670c92e369
I20260808 23:27:32.489001 19508 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93: loading cluster ID for follower catalog manager: success
I20260808 23:27:32.494042 19508 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93: acquiring CA information for follower catalog manager: success
I20260808 23:27:32.496906 19508 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 23:27:32.622711 19350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:32.592370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:39361 (local address 127.0.0.1:55246)
0808 23:27:32.593045 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:32.593071 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:32.593300 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:32.611714 (+ 18414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:32.611720 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:32.611772 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:32.612172 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:32.612193 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.615550 (+ 3357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.615560 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:32.618567 (+ 3007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:32.618601 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.621387 (+ 2786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.621405 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:32.621800 (+ 395us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:32.621899 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 23:27:32.622068 (+ 169us) client_negotiation.cc:240] Negotiation successful
0808 23:27:32.622318 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454}
I20260808 23:27:32.622876 19509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:32.593797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55246 (local address 127.9.44.125:39361)
0808 23:27:32.611030 (+ 17233us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:32.611038 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:32.611069 (+ 31us) server_negotiation.cc:407] Connection header received
0808 23:27:32.611199 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:32.611209 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:32.611348 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:32.611555 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:32.612516 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.615293 (+ 2777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.618864 (+ 3571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:32.621120 (+ 2256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:32.621325 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:32.621710 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:32.622111 (+ 401us) server_negotiation.cc:299] Negotiation successful
0808 23:27:32.622337 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16974,"thread_start_us":16708,"threads_started":1}
I20260808 23:27:32.625108 19444 raft_consensus.cc:1203] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260808 23:27:32.626531 19484 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1653d349ee64235a944e5c763fc4945. Latest consensus state: current_term: 1 leader_uuid: "e1653d349ee64235a944e5c763fc4945" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: NON_VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: true } } }
I20260808 23:27:32.626933 19484 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [sys.catalog]: This master's current role is: LEARNER
W20260808 23:27:32.972824 19378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:32.973331 19378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:32.973694 19378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:32.989183 19378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:27:32.989399 19378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:32.989672 19378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:32.989943 19378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:27:32.990132 19378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:27:33.010070 19378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:40499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:37605,127.9.44.125:39361,127.9.44.124:41697
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.124.keytab
--principal=oryx/127.9.44.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.124:41697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.44.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:33.012607 19378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:33.015215 19378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:33.040859 19516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:33.041988 19515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:33.042047 19518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:33.042290 19378 server_base.cc:1034] running on GCE node
I20260808 23:27:33.043612 19378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:33.045845 19378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:33.047183 19378 hybrid_clock.cc:648] HybridClock initialized: now 1786231653046985 us; error 94 us; skew 500 ppm
I20260808 23:27:33.047928 19378 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:33 dist-test-slave-6bbx krb5kdc[18406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231653, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:33.054973 19378 init.cc:377] Logged in from keytab as oryx/127.9.44.124@KRBTEST.COM (short username oryx)
I20260808 23:27:33.057621 19378 webserver.cc:460] Webserver started at http://127.9.44.124:35307/ using document root <none> and password file <none>
I20260808 23:27:33.058730 19378 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:33.059087 19378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:33.059731 19378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:33.065212 19378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-2/data/instance:
uuid: "8ab17fb16fb04158b064aee7a9998407"
format_stamp: "Formatted at 2026-08-08 23:27:33 on dist-test-slave-6bbx"
server_key: "b9ba8be70256fb0901af0545a33dde43"
server_key_iv: "3eed4c655932a3d18cab6624971ac8bd"
server_key_version: "encryptionkey@0"
I20260808 23:27:33.066505 19378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-2/wal/instance:
uuid: "8ab17fb16fb04158b064aee7a9998407"
format_stamp: "Formatted at 2026-08-08 23:27:33 on dist-test-slave-6bbx"
server_key: "b9ba8be70256fb0901af0545a33dde43"
server_key_iv: "3eed4c655932a3d18cab6624971ac8bd"
server_key_version: "encryptionkey@0"
I20260808 23:27:33.081296 19378 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.014s sys 0.001s
I20260808 23:27:33.090154 19534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:33.092386 19378 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.003s
I20260808 23:27:33.092724 19378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-2/wal
uuid: "8ab17fb16fb04158b064aee7a9998407"
format_stamp: "Formatted at 2026-08-08 23:27:33 on dist-test-slave-6bbx"
server_key: "b9ba8be70256fb0901af0545a33dde43"
server_key_iv: "3eed4c655932a3d18cab6624971ac8bd"
server_key_version: "encryptionkey@0"
I20260808 23:27:33.093160 19378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:33.133304 19378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:33.140693 19543 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e1653d349ee64235a944e5c763fc4945" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: NON_VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: true } } }
I20260808 23:27:33.141508 19543 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:33.143723 19545 raft_consensus.cc:1050] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945: attempting to promote NON_VOTER d6758c943e0b4c7b89118814fd429e93 to VOTER
I20260808 23:27:33.144672 19378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:33.145512 19378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:33.145656 19545 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } }
I20260808 23:27:33.148952 19444 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [term 1 LEARNER]: Refusing update from remote peer e1653d349ee64235a944e5c763fc4945: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 23:27:33.150029 19543 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 23:27:33.156831 19545 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, d6758c943e0b4c7b89118814fd429e93 (127.9.44.125) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } }
I20260808 23:27:33.157732 19444 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, d6758c943e0b4c7b89118814fd429e93 (127.9.44.125) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } }
I20260808 23:27:33.164254 19545 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e1653d349ee64235a944e5c763fc4945" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } }
I20260808 23:27:33.164942 19545 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:33.165742 19548 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e1653d349ee64235a944e5c763fc4945" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } }
I20260808 23:27:33.166823 19548 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:33.169672 19557 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e1653d349ee64235a944e5c763fc4945" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } }
I20260808 23:27:33.170348 19557 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:27:33.219180 19378 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.124:41697
I20260808 23:27:33.219410 19640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.124:41697 every 8 connection(s)
I20260808 23:27:33.221454 19378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-2/data/info.pb
I20260808 23:27:33.225804 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 19378
I20260808 23:27:33.230841 19641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:33.246663 19641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:27:33 dist-test-slave-6bbx krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.44.124@KRBTEST.COM
Aug 08 23:27:33 dist-test-slave-6bbx krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231653, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.124@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM
I20260808 23:27:33.274276 19646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:33.229775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45314 (local address 127.9.44.124:41697)
0808 23:27:33.231591 (+ 1816us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:33.231619 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:33.231738 (+ 119us) server_negotiation.cc:407] Connection header received
0808 23:27:33.235647 (+ 3909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:33.235751 (+ 104us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:33.236408 (+ 657us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:33.237006 (+ 598us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:33.240512 (+ 3506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.243409 (+ 2897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.246228 (+ 2819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.247704 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.247971 (+ 267us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:33.258826 (+ 10855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:33.258839 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:33.258850 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:33.258944 (+ 94us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:33.266527 (+ 7583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:33.268227 (+ 1700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:33.268241 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:33.268250 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:33.268691 (+ 441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:33.269335 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:33.269339 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:33.269342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:33.270434 (+ 1092us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:33.271112 (+ 678us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:33.271835 (+ 723us) server_negotiation.cc:299] Negotiation successful
0808 23:27:33.272307 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1490,"thread_start_us":1098,"threads_started":1}
I20260808 23:27:33.285682 19647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:33.252140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.0.0.1:41982)
0808 23:27:33.254744 (+ 2604us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:33.254837 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:33.255179 (+ 342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:33.256242 (+ 1063us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:33.256249 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:33.257938 (+ 1689us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:33.258960 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:33.258975 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.261857 (+ 2882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.261861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:33.264818 (+ 2957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:33.264840 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.266988 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.267000 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:33.267526 (+ 526us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:33.269005 (+ 1479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:33.269056 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:33.273642 (+ 4586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:33.281337 (+ 7695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:33.281352 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:33.281367 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:33.282106 (+ 739us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:33.282718 (+ 612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:33.282723 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:33.282729 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:33.282932 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:33.283945 (+ 1013us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:33.283956 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:33.284155 (+ 199us) client_negotiation.cc:769] Sending connection context
0808 23:27:33.284385 (+ 230us) client_negotiation.cc:240] Negotiation successful
0808 23:27:33.284868 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2079,"thread_start_us":1677,"threads_started":1}
I20260808 23:27:33.285755 19648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:33.253253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41982 (local address 127.9.44.126:37605)
0808 23:27:33.255694 (+ 2441us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:33.255706 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:33.255764 (+ 58us) server_negotiation.cc:407] Connection header received
0808 23:27:33.255838 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:33.255845 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:33.255984 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:33.256100 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:33.259261 (+ 3161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.261658 (+ 2397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.265057 (+ 3399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.266515 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.266719 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:33.273868 (+ 7149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:33.273874 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:33.273885 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:33.273926 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:33.281130 (+ 7204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:33.282270 (+ 1140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:33.282274 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:33.282276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:33.282564 (+ 288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:33.283085 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:33.283092 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:33.283094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:33.283652 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:33.283942 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:33.284588 (+ 646us) server_negotiation.cc:299] Negotiation successful
0808 23:27:33.284868 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2119,"thread_start_us":1822,"threads_started":1}
I20260808 23:27:33.288204 19641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:27:33 dist-test-slave-6bbx krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231653, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.124@KRBTEST.COM for oryx/127.9.44.125@KRBTEST.COM
I20260808 23:27:33.313642 19647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:33.288966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:39361 (local address 127.0.0.1:55254)
0808 23:27:33.289224 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:33.289241 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:33.289364 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:33.291074 (+ 1710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:33.291089 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:33.291886 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:33.292329 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:33.292343 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.294493 (+ 2150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.294497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:33.296991 (+ 2494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:33.297060 (+ 69us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.298491 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.298498 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:33.298808 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:33.299694 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:33.299763 (+ 69us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:33.303725 (+ 3962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:33.309881 (+ 6156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:33.309885 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:33.309887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:33.310737 (+ 850us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:33.311216 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:33.311221 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:33.311223 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:33.311540 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:33.312348 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:33.312355 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:33.312566 (+ 211us) client_negotiation.cc:769] Sending connection context
0808 23:27:33.312749 (+ 183us) client_negotiation.cc:240] Negotiation successful
0808 23:27:33.313138 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 23:27:33.313999 19649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:33.289265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55254 (local address 127.9.44.125:39361)
0808 23:27:33.290655 (+ 1390us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:33.290660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:33.290684 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:27:33.290764 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:33.290768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:33.290877 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:33.291004 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:33.292537 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.294274 (+ 1737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.297265 (+ 2991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.298280 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.298435 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:33.303960 (+ 5525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:33.303966 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:33.303970 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:33.304008 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:33.309726 (+ 5718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:33.310873 (+ 1147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:33.310877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:33.310879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:33.311105 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:33.311657 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:33.311660 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:33.311707 (+ 47us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:33.312215 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:33.312359 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:33.312921 (+ 562us) server_negotiation.cc:299] Negotiation successful
0808 23:27:33.313253 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1234,"thread_start_us":920,"threads_started":1}
I20260808 23:27:33.315297 19641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.124" port: 41697 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:27:33 dist-test-slave-6bbx krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231653, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.124@KRBTEST.COM for oryx/127.9.44.124@KRBTEST.COM
I20260808 23:27:33.337949 19647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:33.316778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:41697 (local address 127.0.0.1:45322)
0808 23:27:33.317090 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:33.317115 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:33.317235 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:33.317575 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:33.317586 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:33.318270 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:33.318560 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:33.318587 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.320154 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.320158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:33.321959 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:33.321979 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.323430 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.323453 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:33.323912 (+ 459us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:33.324787 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:33.324819 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:33.328413 (+ 3594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:33.334220 (+ 5807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:33.334227 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:33.334229 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:33.335057 (+ 828us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:33.335528 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:33.335540 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:33.335543 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:33.335985 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:33.336753 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:33.336760 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:33.336950 (+ 190us) client_negotiation.cc:769] Sending connection context
0808 23:27:33.337108 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 23:27:33.337382 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 23:27:33.338150 19646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:33.316447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45322 (local address 127.9.44.124:41697)
0808 23:27:33.316670 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:33.316679 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:33.317187 (+ 508us) server_negotiation.cc:407] Connection header received
0808 23:27:33.317352 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:33.317356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:33.317432 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:33.317523 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:33.318716 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.319986 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.322140 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.323174 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.323334 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:33.328593 (+ 5259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:33.328597 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:33.328601 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:33.328642 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:33.334038 (+ 5396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:33.335181 (+ 1143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:33.335185 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:33.335187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:33.335410 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:33.336139 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:33.336144 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:33.336146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:33.336636 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:33.336789 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:33.337272 (+ 483us) server_negotiation.cc:299] Negotiation successful
0808 23:27:33.337563 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":62}
I20260808 23:27:33.347390 19641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407: Bootstrap starting.
I20260808 23:27:33.354615 19641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:33.357599 19641 log.cc:824] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:33.362478 19641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407: No bootstrap required, opened a new log
I20260808 23:27:33.379496 19641 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 41697 } }
I20260808 23:27:33.380244 19641 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:33.380584 19641 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ab17fb16fb04158b064aee7a9998407, State: Initialized, Role: FOLLOWER
I20260808 23:27:33.381551 19641 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 41697 } }
I20260808 23:27:33.384147 19655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 41697 } } }
I20260808 23:27:33.385064 19655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:27:33.385762 19641 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 23:27:33.401854 19553 tablet.cc:2369] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:27:33.409025 19677 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:27:33.409368 19677 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:27:33.415625 19378 master_runner.cc:428] Detected that this master 8ab17fb16fb04158b064aee7a9998407 is joining an existing cluster
I20260808 23:27:33.415972 19378 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.44.124:41697
I20260808 23:27:33.435760 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 18564
I20260808 23:27:33.473559 19685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:33.427108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:41697 (local address 127.0.0.1:45324)
0808 23:27:33.429650 (+ 2542us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:33.429678 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:33.429945 (+ 267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:33.430585 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:33.430590 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:33.432784 (+ 2194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:33.434197 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:33.434229 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.437058 (+ 2829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.437063 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:33.440991 (+ 3928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:33.441006 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.443501 (+ 2495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.443509 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:33.443968 (+ 459us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:33.445203 (+ 1235us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:33.445236 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:33.452877 (+ 7641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:33.465974 (+ 13097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:33.465981 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:33.465989 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:33.467301 (+ 1312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:33.468335 (+ 1034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:33.468340 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:33.468343 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:33.468718 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:33.469772 (+ 1054us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:33.469779 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:33.470532 (+ 753us) client_negotiation.cc:769] Sending connection context
0808 23:27:33.471046 (+ 514us) client_negotiation.cc:240] Negotiation successful
0808 23:27:33.471765 (+ 719us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2198,"thread_start_us":1601,"threads_started":1}
Aug 08 23:27:33 dist-test-slave-6bbx krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.44.65@KRBTEST.COM
I20260808 23:27:33.484694 19646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:33.426676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45324 (local address 127.9.44.124:41697)
0808 23:27:33.427022 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:33.427028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:33.429778 (+ 2750us) server_negotiation.cc:407] Connection header received
0808 23:27:33.430231 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:33.430242 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:33.430366 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:33.430553 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:33.434448 (+ 3895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.436678 (+ 2230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.441198 (+ 4520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.442707 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.442874 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:33.453244 (+ 10370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:33.453258 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:33.453263 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:33.453343 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:33.465791 (+ 12448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:33.467717 (+ 1926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:33.467733 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:33.467737 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:33.468142 (+ 405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:33.468916 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:33.468921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:33.468925 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:33.469624 (+ 699us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:33.472878 (+ 3254us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:33.480345 (+ 7467us) server_negotiation.cc:299] Negotiation successful
0808 23:27:33.480546 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260808 23:27:33.500486 19688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:33.438621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51718 (local address 127.9.44.65:42143)
0808 23:27:33.448833 (+ 10212us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:33.448856 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:33.448905 (+ 49us) server_negotiation.cc:407] Connection header received
0808 23:27:33.449022 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:33.449035 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:33.449399 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:33.449596 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:33.452153 (+ 2557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.455457 (+ 3304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.463712 (+ 8255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.464892 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.465111 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:33.481872 (+ 16761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:33.481897 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:33.481922 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:33.481987 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:33.490618 (+ 8631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:33.492363 (+ 1745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:33.492387 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:33.492394 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:33.492840 (+ 446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:33.493799 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:33.493809 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:33.493812 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:33.494683 (+ 871us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:33.498538 (+ 3855us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:33.498870 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 23:27:33.499331 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9970,"thread_start_us":9649,"threads_started":1}
I20260808 23:27:33.503484 18772 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:33.510318 19686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:33.429902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.0.0.1:41992)
0808 23:27:33.455939 (+ 26037us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:33.455975 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:33.456169 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:33.460373 (+ 4204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:33.460377 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:33.462353 (+ 1976us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:33.466710 (+ 4357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:33.466744 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.469603 (+ 2859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.469612 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:33.487158 (+ 17546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:33.487175 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.490279 (+ 3104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.490299 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:33.490674 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:33.491855 (+ 1181us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:33.491910 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:33.494832 (+ 2922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:33.503612 (+ 8780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:33.503619 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:33.503622 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:33.504498 (+ 876us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:33.505549 (+ 1051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:33.505556 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:33.505559 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:33.505995 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:33.507156 (+ 1161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:33.507162 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:33.507812 (+ 650us) client_negotiation.cc:769] Sending connection context
0808 23:27:33.508064 (+ 252us) client_negotiation.cc:240] Negotiation successful
0808 23:27:33.508760 (+ 696us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25737,"mutex_wait_us":3,"thread_start_us":25317,"threads_started":1}
I20260808 23:27:33.510368 19648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:33.426187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41992 (local address 127.9.44.126:37605)
0808 23:27:33.426472 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:33.426478 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:33.459375 (+ 32897us) server_negotiation.cc:407] Connection header received
0808 23:27:33.459482 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:33.459488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:33.459738 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:33.459887 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:33.467011 (+ 7124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.469240 (+ 2229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.488140 (+ 18900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.489482 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.489637 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:33.495210 (+ 5573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:33.495227 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:33.495245 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:33.495376 (+ 131us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:33.503371 (+ 7995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:33.504746 (+ 1375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:33.504752 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:33.504755 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:33.505085 (+ 330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:33.506222 (+ 1137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:33.506243 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:33.506257 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:33.506958 (+ 701us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:33.507281 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:33.509067 (+ 1786us) server_negotiation.cc:299] Negotiation successful
0808 23:27:33.509491 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 23:27:33.516068 19685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:33.457071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:39361 (local address 127.0.0.1:55266)
0808 23:27:33.474168 (+ 17097us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:33.474190 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:33.474426 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:33.475581 (+ 1155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:33.475594 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:33.477138 (+ 1544us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:33.477494 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:33.477530 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.481693 (+ 4163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.481705 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:33.488442 (+ 6737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:33.488458 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.490949 (+ 2491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.490966 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:33.491932 (+ 966us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:33.493083 (+ 1151us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:33.493131 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:33.495705 (+ 2574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:33.510242 (+ 14537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:33.510256 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:33.510261 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:33.511341 (+ 1080us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:33.512166 (+ 825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:33.512181 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:33.512189 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:33.512510 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:33.513966 (+ 1456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:33.513972 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:33.514308 (+ 336us) client_negotiation.cc:769] Sending connection context
0808 23:27:33.514666 (+ 358us) client_negotiation.cc:240] Negotiation successful
0808 23:27:33.515305 (+ 639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16891,"thread_start_us":22719,"threads_started":1}
I20260808 23:27:33.517836 19649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:33.456786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55266 (local address 127.9.44.125:39361)
0808 23:27:33.460232 (+ 3446us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:33.460237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:33.474297 (+ 14060us) server_negotiation.cc:407] Connection header received
0808 23:27:33.474658 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:33.474681 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:33.475009 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:33.475161 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:33.477840 (+ 2679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.480030 (+ 2190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.488685 (+ 8655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.490286 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.490444 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:33.496108 (+ 5664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:33.496121 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:33.496129 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:33.496203 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:33.506460 (+ 10257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:33.511508 (+ 5048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:33.511518 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:33.511522 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:33.511845 (+ 323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:33.512716 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:33.512728 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:33.512731 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:33.513588 (+ 857us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:33.513804 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:33.516591 (+ 2787us) server_negotiation.cc:299] Negotiation successful
0808 23:27:33.517075 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3270}
I20260808 23:27:33.520668 19655 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:27:33.521382 19655 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 41697 } }
I20260808 23:27:33.524991 19655 leader_election.cc:288] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1653d349ee64235a944e5c763fc4945 (127.9.44.126:37605), d6758c943e0b4c7b89118814fd429e93 (127.9.44.125:39361)
I20260808 23:27:33.525602 19444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8ab17fb16fb04158b064aee7a9998407" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6758c943e0b4c7b89118814fd429e93" is_pre_election: true
I20260808 23:27:33.525270 18481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8ab17fb16fb04158b064aee7a9998407" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1653d349ee64235a944e5c763fc4945" is_pre_election: true
I20260808 23:27:33.527907 19542 leader_election.cc:302] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ab17fb16fb04158b064aee7a9998407; no voters: d6758c943e0b4c7b89118814fd429e93, e1653d349ee64235a944e5c763fc4945
I20260808 23:27:33.529356 19655 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
Aug 08 23:27:33 dist-test-slave-6bbx krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231649, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.126@KRBTEST.COM for oryx/127.9.44.124@KRBTEST.COM
I20260808 23:27:33.549556 19695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:33.523013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:41697 (local address 127.0.0.1:45332)
0808 23:27:33.524472 (+ 1459us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:33.524493 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:33.524864 (+ 371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:33.526087 (+ 1223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:33.526094 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:33.527941 (+ 1847us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:33.528851 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:33.528871 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.531071 (+ 2200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.531074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:33.533280 (+ 2206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:33.533297 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.535076 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.535083 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:33.535437 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:33.536302 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:33.536320 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:33.540621 (+ 4301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:33.545663 (+ 5042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:33.545678 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:33.545682 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:33.546466 (+ 784us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:33.546986 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:33.546993 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:33.546996 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:33.547166 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:33.548154 (+ 988us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:33.548159 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:33.548352 (+ 193us) client_negotiation.cc:769] Sending connection context
0808 23:27:33.548500 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 23:27:33.548890 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1114,"thread_start_us":919,"threads_started":1}
I20260808 23:27:33.549896 19646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:33.523629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45332 (local address 127.9.44.124:41697)
0808 23:27:33.523909 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:33.523918 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:33.524595 (+ 677us) server_negotiation.cc:407] Connection header received
0808 23:27:33.525173 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:33.525180 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:33.525314 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:33.525487 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:33.529192 (+ 3705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.530856 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.533609 (+ 2753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.534819 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.535010 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:33.540902 (+ 5892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:33.540910 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:33.540913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:33.540958 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:33.545503 (+ 4545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:33.546635 (+ 1132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:33.546639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:33.546644 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:33.546847 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:33.547315 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:33.547320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:33.547324 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:33.547961 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:33.548176 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:33.548753 (+ 577us) server_negotiation.cc:299] Negotiation successful
0808 23:27:33.549112 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260808 23:27:33.552407 18464 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.44.124:41697: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "8ab17fb16fb04158b064aee7a9998407"
member_type: NON_VOTER
last_known_addr {
host: "127.9.44.124"
port: 41697
}
attrs {
promote: true
}
}
dest_uuid: "e1653d349ee64235a944e5c763fc4945"
cas_config_opid_index: 6
I20260808 23:27:33.553567 18464 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: NON_VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: true } }
I20260808 23:27:33.559006 19591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:33.559347 19444 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [term 1 FOLLOWER]: Refusing update from remote peer e1653d349ee64235a944e5c763fc4945: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 23:27:33.560938 19543 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 23:27:33.566139 19591 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [term 1 FOLLOWER]: Refusing update from remote peer e1653d349ee64235a944e5c763fc4945: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 23:27:33.566511 19543 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 8ab17fb16fb04158b064aee7a9998407 (127.9.44.124) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: NON_VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: true } } }
I20260808 23:27:33.568257 19444 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 8ab17fb16fb04158b064aee7a9998407 (127.9.44.124) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: NON_VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: true } } }
I20260808 23:27:33.567826 19548 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: NON_VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:27:33.577492 19557 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e1653d349ee64235a944e5c763fc4945" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: NON_VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: true } } }
I20260808 23:27:33.577903 19557 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:27:33.582367 19543 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.44.124:41697
I20260808 23:27:33.583261 19378 master.cc:537] Master@127.9.44.124:41697 shutting down...
I20260808 23:27:33.583124 19543 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e1653d349ee64235a944e5c763fc4945" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: NON_VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: true } } }
I20260808 23:27:33.584290 19543 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:33.584645 19548 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e1653d349ee64235a944e5c763fc4945" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: NON_VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: true } } }
I20260808 23:27:33.585132 19548 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:33.624473 19591 raft_consensus.cc:858] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 41697 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: NON_VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } }
W20260808 23:27:33.625043 19591 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 1 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 41697 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: NON_VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } }
I20260808 23:27:33.639369 19591 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER d6758c943e0b4c7b89118814fd429e93 (127.9.44.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: NON_VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: true } } }
I20260808 23:27:33.646785 19704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e1653d349ee64235a944e5c763fc4945" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: NON_VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: true } } }
I20260808 23:27:33.647647 19704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 23:27:33.648228 19701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e1653d349ee64235a944e5c763fc4945" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: NON_VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: true } } }
I20260808 23:27:33.649224 19701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 23:27:33.649677 19655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1653d349ee64235a944e5c763fc4945. Latest consensus state: current_term: 1 leader_uuid: "e1653d349ee64235a944e5c763fc4945" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: NON_VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: true } } }
I20260808 23:27:33.650135 19655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 23:27:33.650911 19543 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e1653d349ee64235a944e5c763fc4945" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: NON_VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: true } } }
I20260808 23:27:33.651499 19543 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:33.656251 19378 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260808 23:27:33.657001 19378 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260808 23:27:33.657302 19378 tablet_replica.cc:330] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407: stopping tablet replica
W20260808 23:27:33.657272 18450 consensus_peers.cc:487] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 -> Peer 8ab17fb16fb04158b064aee7a9998407 (127.9.44.124:41697): Couldn't send request to peer 8ab17fb16fb04158b064aee7a9998407. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 23:27:33.678871 19378 master.cc:552] Master@127.9.44.124:41697 shutdown complete.
I20260808 23:27:33.679040 19378 master_runner.cc:305] Clearing existing system tablet
I20260808 23:27:33.680110 19378 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:33.680687 19378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:33.686730 19378 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 23:27:33.690588 19713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:33.691277 19378 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:33.693986 19378 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.003s
I20260808 23:27:33.694244 19378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-2/wal
uuid: "8ab17fb16fb04158b064aee7a9998407"
format_stamp: "Formatted at 2026-08-08 23:27:33 on dist-test-slave-6bbx"
server_key: "b9ba8be70256fb0901af0545a33dde43"
server_key_iv: "3eed4c655932a3d18cab6624971ac8bd"
server_key_version: "encryptionkey@0"
I20260808 23:27:33.696163 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 18564
I20260808 23:27:33.703035 19378 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:27:33.715355 19378 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 23:27:33.715715 19378 log.cc:1196] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 23:27:33.716786 19378 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407: Deleting consensus metadata
I20260808 23:27:33.717314 19378 master_runner.cc:315] Copying system tablet from 127.9.44.126:37605
I20260808 23:27:33.726899 19378 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407: tablet copy: Beginning tablet copy session from remote peer at address 127.9.44.126:37605
I20260808 23:27:33.741101 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:42143
--local_ip_for_outbound_sockets=127.9.44.65
--tserver_master_addrs=127.9.44.126:37605
--webserver_port=35237
--webserver_interface=127.9.44.65
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.65.keytab
--principal=oryx/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:40499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.44.126:37605,127.9.44.125:39361
--tserver_master_addrs=127.9.44.126:37605,127.9.44.125:39361,127.9.44.124:41697 with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:27:33.753852 19648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:33.728322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42000 (local address 127.9.44.126:37605)
0808 23:27:33.728688 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:33.728694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:33.736672 (+ 7978us) server_negotiation.cc:407] Connection header received
0808 23:27:33.737193 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:33.737203 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:33.737345 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:33.737471 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:33.739572 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.740622 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.742663 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.743280 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.743480 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:33.746231 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:33.746242 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:33.746246 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:33.746298 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:33.749912 (+ 3614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:33.750770 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:33.750783 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:33.750787 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:33.751026 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:33.751556 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:33.751562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:33.751570 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:33.752026 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:33.752247 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:33.752831 (+ 584us) server_negotiation.cc:299] Negotiation successful
0808 23:27:33.753169 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182}
I20260808 23:27:33.753880 19721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:33.728314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.0.0.1:42000)
0808 23:27:33.736483 (+ 8169us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:33.736532 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:33.736900 (+ 368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:33.737917 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:33.737928 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:33.738880 (+ 952us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:33.739286 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:33.739308 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.740840 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.740847 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:33.742439 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:33.742458 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:33.743499 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:33.743508 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:33.743846 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:33.744674 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:33.744712 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:33.745992 (+ 1280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:33.750079 (+ 4087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:33.750092 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:33.750096 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:33.750618 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:33.751178 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:33.751189 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:33.751193 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:33.751414 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:33.752181 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:33.752188 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:33.752467 (+ 279us) client_negotiation.cc:769] Sending connection context
0808 23:27:33.752671 (+ 204us) client_negotiation.cc:240] Negotiation successful
0808 23:27:33.753193 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7925,"thread_start_us":7664,"threads_started":1}
I20260808 23:27:33.755312 18499 tablet_copy_service.cc:140] P e1653d349ee64235a944e5c763fc4945: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 8ab17fb16fb04158b064aee7a9998407 ({username='oryx', principal='oryx/127.9.44.124@KRBTEST.COM'} at 127.0.0.1:42000)
I20260808 23:27:33.755576 18499 tablet_copy_service.cc:161] P e1653d349ee64235a944e5c763fc4945: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 8ab17fb16fb04158b064aee7a9998407 at {username='oryx', principal='oryx/127.9.44.124@KRBTEST.COM'} at 127.0.0.1:42000: session id = 8ab17fb16fb04158b064aee7a9998407-00000000000000000000000000000000
I20260808 23:27:33.761255 18499 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 23:27:33.764256 19378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 23:27:33.775820 19378 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407: tablet copy: Starting download of 0 data blocks...
I20260808 23:27:33.776327 19378 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407: tablet copy: Starting download of 1 WAL segments...
I20260808 23:27:33.780494 19378 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 23:27:33.787192 18499 tablet_copy_service.cc:342] P e1653d349ee64235a944e5c763fc4945: Request end of tablet copy session 8ab17fb16fb04158b064aee7a9998407-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.44.124@KRBTEST.COM'} at 127.0.0.1:42000
I20260808 23:27:33.787411 18499 tablet_copy_service.cc:433] P e1653d349ee64235a944e5c763fc4945: ending tablet copy session 8ab17fb16fb04158b064aee7a9998407-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 8ab17fb16fb04158b064aee7a9998407
I20260808 23:27:33.806361 19378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:33.810070 19726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:33.810786 19727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:33.819790 19378 server_base.cc:1034] running on GCE node
W20260808 23:27:33.819973 19729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:33.820683 19378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:33.821815 19378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:33.823136 19378 hybrid_clock.cc:648] HybridClock initialized: now 1786231653823064 us; error 53 us; skew 500 ppm
I20260808 23:27:33.823606 19378 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:33 dist-test-slave-6bbx krb5kdc[18406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231653, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:33.826674 19378 init.cc:377] Logged in from keytab as oryx/127.9.44.124@KRBTEST.COM (short username oryx)
I20260808 23:27:33.833362 19378 webserver.cc:460] Webserver started at http://127.9.44.124:45801/ using document root <none> and password file <none>
I20260808 23:27:33.834015 19378 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:33.834275 19378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:33.839701 19378 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 23:27:33.843124 19741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:33.843859 19378 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 23:27:33.844115 19378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-2/wal
uuid: "8ab17fb16fb04158b064aee7a9998407"
format_stamp: "Formatted at 2026-08-08 23:27:33 on dist-test-slave-6bbx"
server_key: "b9ba8be70256fb0901af0545a33dde43"
server_key_iv: "3eed4c655932a3d18cab6624971ac8bd"
server_key_version: "encryptionkey@0"
I20260808 23:27:33.844552 19378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:33.878371 19378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:33.883939 19378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:33.936199 19378 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.124:41697
I20260808 23:27:33.936259 19836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.124:41697 every 8 connection(s)
I20260808 23:27:33.938200 19378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/master-2/data/info.pb
I20260808 23:27:33.947332 19838 sys_catalog.cc:263] Verifying existing consensus state
I20260808 23:27:33.953158 19838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407: Bootstrap starting.
I20260808 23:27:33.977790 19838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:27:33.978549 19838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407: Bootstrap complete.
I20260808 23:27:33.980232 19838 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: NON_VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: true } }
I20260808 23:27:33.980736 19838 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8ab17fb16fb04158b064aee7a9998407, State: Initialized, Role: LEARNER
I20260808 23:27:33.981134 19838 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: NON_VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: true } }
I20260808 23:27:33.982932 19838 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:27:33.982496 19844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: NON_VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: true } } }
I20260808 23:27:33.983299 19844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [sys.catalog]: This master's current role is: LEARNER
I20260808 23:27:34.004122 19862 catalog_manager.cc:1259] Loaded cluster ID: 072e9d2097594b749e6fc0670c92e369
I20260808 23:27:34.004297 19862 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407: loading cluster ID for follower catalog manager: success
I20260808 23:27:34.010619 19862 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407: acquiring CA information for follower catalog manager: success
I20260808 23:27:34.015619 19862 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 23:27:34.272830 19868 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e1653d349ee64235a944e5c763fc4945" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: NON_VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: true } } }
I20260808 23:27:34.274777 19868 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:34.300366 19873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:34.266311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:41697 (local address 127.0.0.1:45342)
0808 23:27:34.280952 (+ 14641us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:34.281001 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:34.281278 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:34.283725 (+ 2447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:34.283741 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:34.283793 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:34.284507 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:34.284536 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:34.289844 (+ 5308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:34.289853 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:34.294221 (+ 4368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:34.294258 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:34.298743 (+ 4485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:34.298753 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:34.299146 (+ 393us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:34.299221 (+ 75us) client_negotiation.cc:769] Sending connection context
0808 23:27:34.299404 (+ 183us) client_negotiation.cc:240] Negotiation successful
0808 23:27:34.299649 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14376,"thread_start_us":15123,"threads_started":1}
I20260808 23:27:34.301185 19874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:34.267542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45342 (local address 127.9.44.124:41697)
0808 23:27:34.282578 (+ 15036us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:34.282596 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:34.282627 (+ 31us) server_negotiation.cc:407] Connection header received
0808 23:27:34.282836 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:34.282842 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:34.283066 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:34.283287 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:34.285047 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:34.289392 (+ 4345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:34.294522 (+ 5130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:34.297735 (+ 3213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:34.298033 (+ 298us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:34.298556 (+ 523us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:34.299586 (+ 1030us) server_negotiation.cc:299] Negotiation successful
0808 23:27:34.299989 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14798,"thread_start_us":14570,"threads_started":1}
I20260808 23:27:34.302942 19787 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.5->[1.6-1.7] Dedup: 1.7->[]
I20260808 23:27:34.303776 19844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1653d349ee64235a944e5c763fc4945. Latest consensus state: current_term: 1 leader_uuid: "e1653d349ee64235a944e5c763fc4945" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: NON_VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: true } } }
I20260808 23:27:34.304360 19844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [sys.catalog]: This master's current role is: LEARNER
I20260808 23:27:34.306087 19869 raft_consensus.cc:1050] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945: attempting to promote NON_VOTER 8ab17fb16fb04158b064aee7a9998407 to VOTER
I20260808 23:27:34.307386 19869 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: false } }
I20260808 23:27:34.312812 19787 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [term 1 LEARNER]: Refusing update from remote peer e1653d349ee64235a944e5c763fc4945: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260808 23:27:34.314152 19872 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260808 23:27:34.318755 19444 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [term 1 FOLLOWER]: Refusing update from remote peer e1653d349ee64235a944e5c763fc4945: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260808 23:27:34.319950 19868 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260808 23:27:34.326475 19868 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 8ab17fb16fb04158b064aee7a9998407 (127.9.44.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: false } } }
I20260808 23:27:34.338308 19787 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 8ab17fb16fb04158b064aee7a9998407 (127.9.44.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: false } } }
I20260808 23:27:34.340209 19868 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e1653d349ee64235a944e5c763fc4945" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: false } } }
I20260808 23:27:34.340952 19868 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:34.338500 19444 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 8ab17fb16fb04158b064aee7a9998407 (127.9.44.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: false } } }
I20260808 23:27:34.341365 19878 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e1653d349ee64235a944e5c763fc4945" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: false } } }
I20260808 23:27:34.342566 19878 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:34.343464 19868 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e1653d349ee64235a944e5c763fc4945" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: false } } }
I20260808 23:27:34.344139 19868 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:34.350947 19844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e1653d349ee64235a944e5c763fc4945" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: false } } }
I20260808 23:27:34.351692 19844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:27:34.362663 19880 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e1653d349ee64235a944e5c763fc4945" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: false } } }
I20260808 23:27:34.363240 19880 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 23:27:34.549842 19722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:34.550450 19722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:34.550834 19722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:34.568249 19722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:34.568562 19722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:34.568969 19722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:27:34.591168 19722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:40499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.65.keytab
--principal=oryx/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:42143
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=35237
--tserver_master_addrs=127.9.44.126:37605,127.9.44.125:39361,127.9.44.124:41697
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:34.594007 19722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:34.596949 19722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:34.631371 19887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:34.631488 19889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:34.631636 19892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:34.637907 19722 server_base.cc:1034] running on GCE node
I20260808 23:27:34.639565 19722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:34.641963 19722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:34.643508 19722 hybrid_clock.cc:648] HybridClock initialized: now 1786231654643204 us; error 95 us; skew 500 ppm
I20260808 23:27:34.644239 19722 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:34 dist-test-slave-6bbx krb5kdc[18406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231654, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:34.652669 19722 init.cc:377] Logged in from keytab as oryx/127.9.44.65@KRBTEST.COM (short username oryx)
I20260808 23:27:34.657208 19722 webserver.cc:460] Webserver started at http://127.9.44.65:35237/ using document root <none> and password file <none>
I20260808 23:27:34.659024 19722 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:34.659488 19722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:34.674216 19722 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.003s sys 0.004s
I20260808 23:27:34.681952 19901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:34.685400 19722 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.005s sys 0.000s
I20260808 23:27:34.685804 19722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2"
format_stamp: "Formatted at 2026-08-08 23:27:30 on dist-test-slave-6bbx"
server_key: "0def10741324d574f44131c463dbce38"
server_key_iv: "b01c5a8cabacee80d2fd032d0ac18839"
server_key_version: "encryptionkey@0"
I20260808 23:27:34.688750 19722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:34.722376 19722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:34.734375 19722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:34.735251 19722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:34.738651 19722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:34.742936 19722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:34.743335 19722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:34.743592 19722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:34.743999 19722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.002s
I20260808 23:27:34.904273 19722 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:42143
I20260808 23:27:34.904690 20081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:42143 every 8 connection(s)
I20260808 23:27:34.908563 19722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:27:34.913123 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 19722
I20260808 23:27:34.914631 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 18792
Aug 08 23:27:34 dist-test-slave-6bbx krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.44.66@KRBTEST.COM
Aug 08 23:27:35 dist-test-slave-6bbx krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231654, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.124@KRBTEST.COM
Aug 08 23:27:35 dist-test-slave-6bbx krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231654, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.125@KRBTEST.COM
Aug 08 23:27:35 dist-test-slave-6bbx krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231654, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM
I20260808 23:27:35.029981 20090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:34.918096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40280 (local address 127.9.44.66:39169)
0808 23:27:34.922158 (+ 4062us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:34.922199 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:34.955200 (+ 33001us) server_negotiation.cc:407] Connection header received
0808 23:27:34.956149 (+ 949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:34.956160 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:34.957082 (+ 922us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:34.957565 (+ 483us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:34.965847 (+ 8282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:34.971547 (+ 5700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:34.976954 (+ 5407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:34.979882 (+ 2928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:34.980082 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:35.001309 (+ 21227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:35.001361 (+ 52us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:35.001398 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:35.001546 (+ 148us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:35.014525 (+ 12979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:35.020192 (+ 5667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:35.020204 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:35.020208 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:35.020667 (+ 459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:35.021978 (+ 1311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:35.021992 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:35.021995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:35.023581 (+ 1586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:35.027408 (+ 3827us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:35.027901 (+ 493us) server_negotiation.cc:299] Negotiation successful
0808 23:27:35.028586 (+ 685us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3413,"thread_start_us":2673,"threads_started":1}
I20260808 23:27:35.033555 18992 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:35.044117 20097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:34.922300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:42731 (local address 127.9.44.126:37605)
0808 23:27:34.959230 (+ 36930us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:34.959258 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:34.959337 (+ 79us) server_negotiation.cc:407] Connection header received
0808 23:27:34.985212 (+ 25875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:34.985265 (+ 53us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:34.985599 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:34.985881 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:34.995045 (+ 9164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:34.997837 (+ 2792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:35.006052 (+ 8215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:35.007614 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:35.007844 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:35.022449 (+ 14605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:35.022469 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:35.022484 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:35.022563 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:35.032679 (+ 10116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:35.034529 (+ 1850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:35.034537 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:35.034542 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:35.034939 (+ 397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:35.038712 (+ 3773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:35.038718 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:35.038732 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:35.039544 (+ 812us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:35.039807 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:35.042285 (+ 2478us) server_negotiation.cc:299] Negotiation successful
0808 23:27:35.042645 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36561,"thread_start_us":36253,"threads_started":1}
I20260808 23:27:35.047333 20095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:34.920255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:33553 (local address 127.9.44.124:41697)
0808 23:27:34.945352 (+ 25097us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:34.945371 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:34.962625 (+ 17254us) server_negotiation.cc:407] Connection header received
0808 23:27:34.985461 (+ 22836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:34.985480 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:34.985656 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:34.985960 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:34.994850 (+ 8890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:34.998379 (+ 3529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:35.006035 (+ 7656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:35.008068 (+ 2033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:35.008340 (+ 272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:35.020589 (+ 12249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:35.020601 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:35.020618 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:35.020717 (+ 99us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:35.030482 (+ 9765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:35.038768 (+ 8286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:35.038781 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:35.038783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:35.039062 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:35.040439 (+ 1377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:35.040448 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:35.040452 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:35.041565 (+ 1113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:35.042084 (+ 519us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:35.044090 (+ 2006us) server_negotiation.cc:299] Negotiation successful
0808 23:27:35.044878 (+ 788us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8551,"thread_start_us":8121,"threads_started":1}
I20260808 23:27:35.047376 20100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:34.926649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.9.44.65:42731)
0808 23:27:34.953963 (+ 27314us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:34.954156 (+ 193us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:34.984389 (+ 30233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:34.987240 (+ 2851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:34.987291 (+ 51us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:34.990278 (+ 2987us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:34.994428 (+ 4150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:34.994450 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:34.998305 (+ 3855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:34.998323 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:35.005647 (+ 7324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:35.005678 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:35.008833 (+ 3155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:35.008845 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:35.009558 (+ 713us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:35.011218 (+ 1660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:35.011384 (+ 166us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:35.021874 (+ 10490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:35.032888 (+ 11014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:35.032905 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:35.032908 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:35.034310 (+ 1402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:35.035119 (+ 809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:35.035125 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:35.035128 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:35.038488 (+ 3360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:35.040449 (+ 1961us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:35.040468 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:35.041388 (+ 920us) client_negotiation.cc:769] Sending connection context
0808 23:27:35.042087 (+ 699us) client_negotiation.cc:240] Negotiation successful
0808 23:27:35.043293 (+ 1206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25925,"mutex_wait_us":465,"thread_start_us":40584,"threads_started":1}
I20260808 23:27:35.048280 20103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:34.969692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:51121 (local address 127.9.44.125:39361)
0808 23:27:34.982309 (+ 12617us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:34.982316 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:34.982349 (+ 33us) server_negotiation.cc:407] Connection header received
0808 23:27:34.990676 (+ 8327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:34.990708 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:34.990992 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:34.991326 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:34.997677 (+ 6351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:35.000718 (+ 3041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:35.006383 (+ 5665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:35.008072 (+ 1689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:35.008339 (+ 267us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:35.021353 (+ 13014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:35.021373 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:35.021380 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:35.021482 (+ 102us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:35.031357 (+ 9875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:35.038983 (+ 7626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:35.038999 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:35.039004 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:35.039357 (+ 353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:35.040802 (+ 1445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:35.040811 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:35.040814 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:35.042181 (+ 1367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:35.042374 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:35.043988 (+ 1614us) server_negotiation.cc:299] Negotiation successful
0808 23:27:35.045024 (+ 1036us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12411,"thread_start_us":12630,"threads_started":1}
I20260808 23:27:35.048934 20098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:34.929188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:41697 (local address 127.9.44.65:33553)
0808 23:27:34.962351 (+ 33163us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:34.962421 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:34.984389 (+ 21968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:34.987240 (+ 2851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:34.987291 (+ 51us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:34.989998 (+ 2707us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:34.994050 (+ 4052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:34.994143 (+ 93us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:34.998942 (+ 4799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:34.998960 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:35.005635 (+ 6675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:35.005665 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:35.009264 (+ 3599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:35.009286 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:35.009843 (+ 557us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:35.011481 (+ 1638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:35.011531 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:35.019975 (+ 8444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:35.030791 (+ 10816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:35.030889 (+ 98us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:35.030930 (+ 41us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:35.038572 (+ 7642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:35.039524 (+ 952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:35.039533 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:35.039537 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:35.040213 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:35.042234 (+ 2021us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:35.042241 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:35.042450 (+ 209us) client_negotiation.cc:769] Sending connection context
0808 23:27:35.042844 (+ 394us) client_negotiation.cc:240] Negotiation successful
0808 23:27:35.046406 (+ 3562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32631,"mutex_wait_us":3205,"thread_start_us":19703,"threads_started":1}
I20260808 23:27:35.049440 20104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:34.975292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:39361 (local address 127.9.44.65:51121)
0808 23:27:34.977582 (+ 2290us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:34.977622 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:34.988663 (+ 11041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:34.992221 (+ 3558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:34.992227 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:34.995427 (+ 3200us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:34.997301 (+ 1874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:34.997320 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:35.001047 (+ 3727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:35.001061 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:35.006126 (+ 5065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:35.006149 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:35.010942 (+ 4793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:35.010961 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:35.011605 (+ 644us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:35.013074 (+ 1469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:35.013117 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:35.020569 (+ 7452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:35.031729 (+ 11160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:35.031743 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:35.031749 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:35.038704 (+ 6955us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:35.039638 (+ 934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:35.039647 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:35.039651 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:35.040420 (+ 769us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:35.042591 (+ 2171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:35.042598 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:35.043063 (+ 465us) client_negotiation.cc:769] Sending connection context
0808 23:27:35.043413 (+ 350us) client_negotiation.cc:240] Negotiation successful
0808 23:27:35.046052 (+ 2639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1976,"thread_start_us":1401,"threads_started":1}
I20260808 23:27:35.051652 20082 heartbeater.cc:347] Connected to a master server at 127.9.44.126:37605
I20260808 23:27:35.052004 20085 heartbeater.cc:347] Connected to a master server at 127.9.44.124:41697
I20260808 23:27:35.052539 20082 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:35.052573 20085 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:35.054490 20088 heartbeater.cc:347] Connected to a master server at 127.9.44.125:39361
I20260808 23:27:35.054940 20088 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:35.055415 20085 heartbeater.cc:511] Master 127.9.44.124:41697 requested a full tablet report, sending...
I20260808 23:27:35.056941 20082 heartbeater.cc:511] Master 127.9.44.126:37605 requested a full tablet report, sending...
I20260808 23:27:35.057291 20088 heartbeater.cc:511] Master 127.9.44.125:39361 requested a full tablet report, sending...
I20260808 23:27:35.060065 18464 ts_manager.cc:194] Re-registered known tserver with Master: f1ac6dffcc5c43dcaaec28cb6b1130f2 (127.9.44.65:42143)
I20260808 23:27:35.061878 18464 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.44.65@KRBTEST.COM'} at 127.9.44.65:42731
I20260808 23:27:35.062856 19773 ts_manager.cc:194] Registered new tserver with Master: f1ac6dffcc5c43dcaaec28cb6b1130f2 (127.9.44.65:42143)
I20260808 23:27:35.064939 19426 ts_manager.cc:194] Registered new tserver with Master: f1ac6dffcc5c43dcaaec28cb6b1130f2 (127.9.44.65:42143)
I20260808 23:27:35.182381 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 18792
I20260808 23:27:35.206429 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:39169
--local_ip_for_outbound_sockets=127.9.44.66
--tserver_master_addrs=127.9.44.126:37605
--webserver_port=42901
--webserver_interface=127.9.44.66
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.66.keytab
--principal=oryx/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:40499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.44.126:37605,127.9.44.125:39361
--tserver_master_addrs=127.9.44.126:37605,127.9.44.125:39361,127.9.44.124:41697 with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:35.916162 20108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:35.916533 20108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:35.916795 20108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:35.928618 20108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:35.928845 20108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:35.929100 20108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:27:35.946983 20108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:40499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.66.keytab
--principal=oryx/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:39169
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=42901
--tserver_master_addrs=127.9.44.126:37605,127.9.44.125:39361,127.9.44.124:41697
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:35.948606 20108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:35.950227 20108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:35.970002 20116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:35.970466 20120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:35.970993 20117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:35.971800 20108 server_base.cc:1034] running on GCE node
I20260808 23:27:35.972395 20108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:35.973387 20108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:35.974659 20108 hybrid_clock.cc:648] HybridClock initialized: now 1786231655974552 us; error 41 us; skew 500 ppm
I20260808 23:27:35.975098 20108 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:35 dist-test-slave-6bbx krb5kdc[18406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231655, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:35.979943 20108 init.cc:377] Logged in from keytab as oryx/127.9.44.66@KRBTEST.COM (short username oryx)
I20260808 23:27:35.982313 20108 webserver.cc:460] Webserver started at http://127.9.44.66:42901/ using document root <none> and password file <none>
I20260808 23:27:35.983273 20108 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:35.983510 20108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:35.990006 20108 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 23:27:35.993863 20133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:35.995321 20108 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260808 23:27:35.995534 20108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "a140e60268f342a29eff122f628bbab5"
format_stamp: "Formatted at 2026-08-08 23:27:30 on dist-test-slave-6bbx"
server_key: "967a45a35b06ef25ddfbf97c3fa03abc"
server_key_iv: "f002cb6a5ff17c4c95441ae4ee67bca9"
server_key_version: "encryptionkey@0"
I20260808 23:27:35.996817 20108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:36.021106 20108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:36.026926 20108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:36.027251 20108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:36.028278 20108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:36.030542 20108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:36.030733 20108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:36.030889 20108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:36.031064 20108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:36.068034 20082 heartbeater.cc:503] Master 127.9.44.126:37605 was elected leader, sending a full tablet report...
I20260808 23:27:36.110389 20108 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:39169
I20260808 23:27:36.110423 20323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:39169 every 8 connection(s)
I20260808 23:27:36.111975 20108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:27:36.116034 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 20108
I20260808 23:27:36.116317 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 19008
Aug 08 23:27:36 dist-test-slave-6bbx krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231655, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.124@KRBTEST.COM
Aug 08 23:27:36 dist-test-slave-6bbx krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.44.67@KRBTEST.COM
Aug 08 23:27:36 dist-test-slave-6bbx krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231655, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM
I20260808 23:27:36.141055 20333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.115171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:49371 (local address 127.9.44.124:41697)
0808 23:27:36.115596 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:36.115601 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:36.116992 (+ 1391us) server_negotiation.cc:407] Connection header received
0808 23:27:36.120995 (+ 4003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:36.121001 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:36.121105 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:36.121212 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:36.123780 (+ 2568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.124966 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.127490 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.128215 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.128342 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.134031 (+ 5689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:36.134040 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:36.134045 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:36.134083 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:36.137777 (+ 3694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:36.138654 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:36.138660 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:36.138664 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:36.138842 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:36.139248 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:36.139252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:36.139254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:36.139522 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:36.139649 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:36.140467 (+ 818us) server_negotiation.cc:299] Negotiation successful
0808 23:27:36.140703 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":204,"threads_started":1}
I20260808 23:27:36.142030 20337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.115593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:41697 (local address 127.9.44.66:49371)
0808 23:27:36.116829 (+ 1236us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:36.116889 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:36.120532 (+ 3643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:36.121842 (+ 1310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:36.121855 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:36.122659 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:36.123522 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:36.123547 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.125290 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.125296 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:36.127319 (+ 2023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:36.127335 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.128673 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.128680 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:36.129013 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.129717 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:36.129755 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:36.133709 (+ 3954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:36.137930 (+ 4221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:36.137939 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:36.137953 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:36.138508 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:36.138940 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:36.138945 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:36.138949 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:36.139129 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:36.139644 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:36.139652 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:36.140039 (+ 387us) client_negotiation.cc:769] Sending connection context
0808 23:27:36.140343 (+ 304us) client_negotiation.cc:240] Negotiation successful
0808 23:27:36.140803 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":900,"thread_start_us":840,"threads_started":1}
I20260808 23:27:36.142927 20344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.117319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54366 (local address 127.9.44.67:33679)
0808 23:27:36.117909 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:36.117915 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:36.117936 (+ 21us) server_negotiation.cc:407] Connection header received
0808 23:27:36.117993 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:36.117998 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:36.118209 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:36.118352 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:36.120611 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.122468 (+ 1857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.124768 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.125678 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.126467 (+ 789us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.134478 (+ 8011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:36.134493 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:36.134508 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:36.134550 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:36.138665 (+ 4115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:36.139515 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:36.139524 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:36.139528 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:36.139728 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:36.140173 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:36.140178 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:36.140182 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:36.140594 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:36.142350 (+ 1756us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:36.142469 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 23:27:36.142653 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":279,"threads_started":1}
I20260808 23:27:36.143764 20326 heartbeater.cc:347] Connected to a master server at 127.9.44.124:41697
I20260808 23:27:36.144084 20326 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:36.145064 20326 heartbeater.cc:511] Master 127.9.44.124:41697 requested a full tablet report, sending...
Aug 08 23:27:36 dist-test-slave-6bbx krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231655, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.125@KRBTEST.COM
I20260808 23:27:36.146219 19201 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:36.146790 19773 ts_manager.cc:194] Registered new tserver with Master: a140e60268f342a29eff122f628bbab5 (127.9.44.66:39169)
I20260808 23:27:36.149024 20338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.116046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.9.44.66:44101)
0808 23:27:36.117966 (+ 1920us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:36.117983 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:36.127470 (+ 9487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:36.127835 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:36.127839 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:36.128382 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:36.128735 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:36.128749 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.129974 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.129979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:36.131548 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:36.131564 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.133933 (+ 2369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.133940 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:36.134099 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.134720 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:36.134745 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:36.137325 (+ 2580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:36.142407 (+ 5082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:36.142412 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:36.142415 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:36.142936 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:36.147313 (+ 4377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:36.147318 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:36.147321 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:36.147460 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:36.148049 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:36.148054 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:36.148210 (+ 156us) client_negotiation.cc:769] Sending connection context
0808 23:27:36.148371 (+ 161us) client_negotiation.cc:240] Negotiation successful
0808 23:27:36.148632 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1794,"spinlock_wait_cycles":52096,"thread_start_us":1044,"threads_started":1}
I20260808 23:27:36.149036 20330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.114765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:44101 (local address 127.9.44.126:37605)
0808 23:27:36.115394 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:36.115401 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:36.118081 (+ 2680us) server_negotiation.cc:407] Connection header received
0808 23:27:36.127591 (+ 9510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:36.127599 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:36.127710 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:36.127819 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:36.128876 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.129842 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.133080 (+ 3238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.133763 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.133899 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.137531 (+ 3632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:36.137538 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:36.137543 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:36.137582 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:36.141027 (+ 3445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:36.143139 (+ 2112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:36.143145 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:36.143148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:36.143299 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:36.147581 (+ 4282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:36.147586 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:36.147589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:36.147928 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:36.148066 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:36.148484 (+ 418us) server_negotiation.cc:299] Negotiation successful
0808 23:27:36.148708 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":380,"threads_started":1}
I20260808 23:27:36.149894 20324 heartbeater.cc:347] Connected to a master server at 127.9.44.126:37605
I20260808 23:27:36.150002 20324 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:36.150321 20324 heartbeater.cc:511] Master 127.9.44.126:37605 requested a full tablet report, sending...
I20260808 23:27:36.151408 18464 ts_manager.cc:194] Re-registered known tserver with Master: a140e60268f342a29eff122f628bbab5 (127.9.44.66:39169)
I20260808 23:27:36.152495 18464 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.44.66@KRBTEST.COM'} at 127.9.44.66:44101
I20260808 23:27:36.153859 20346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.119206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:36521 (local address 127.9.44.125:39361)
0808 23:27:36.132124 (+ 12918us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:36.132131 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:36.132237 (+ 106us) server_negotiation.cc:407] Connection header received
0808 23:27:36.132305 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:36.132311 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:36.132389 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:36.132503 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:36.135252 (+ 2749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.136343 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.141187 (+ 4844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.141796 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.141921 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.146882 (+ 4961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:36.146889 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:36.146894 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:36.146932 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:36.151023 (+ 4091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:36.151812 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:36.151818 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:36.151821 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:36.151967 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:36.152407 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:36.152412 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:36.152416 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:36.152742 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:36.152865 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:36.153392 (+ 527us) server_negotiation.cc:299] Negotiation successful
0808 23:27:36.153602 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12795,"thread_start_us":12645,"threads_started":1}
I20260808 23:27:36.153909 20345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.119054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:39361 (local address 127.9.44.66:36521)
0808 23:27:36.129058 (+ 10004us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:36.129075 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:36.129178 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:36.133132 (+ 3954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:36.133137 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:36.134329 (+ 1192us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:36.134744 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:36.134759 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.136508 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.136514 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:36.138723 (+ 2209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:36.138736 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.142004 (+ 3268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.142012 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:36.143412 (+ 1400us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.143972 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:36.143995 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:36.146688 (+ 2693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:36.151200 (+ 4512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:36.151204 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:36.151207 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:36.151644 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:36.152107 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:36.152113 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:36.152117 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:36.152285 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:36.152978 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:36.152983 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:36.153120 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 23:27:36.153285 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 23:27:36.153534 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9886,"thread_start_us":12576,"threads_started":1}
I20260808 23:27:36.154978 20327 heartbeater.cc:347] Connected to a master server at 127.9.44.125:39361
I20260808 23:27:36.155088 20327 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:36.155417 20327 heartbeater.cc:511] Master 127.9.44.125:39361 requested a full tablet report, sending...
I20260808 23:27:36.156368 19426 ts_manager.cc:194] Registered new tserver with Master: a140e60268f342a29eff122f628bbab5 (127.9.44.66:39169)
I20260808 23:27:36.233415 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 19008
I20260808 23:27:36.257377 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:33679
--local_ip_for_outbound_sockets=127.9.44.67
--tserver_master_addrs=127.9.44.126:37605
--webserver_port=35655
--webserver_interface=127.9.44.67
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.67.keytab
--principal=oryx/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:40499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.44.126:37605,127.9.44.125:39361
--tserver_master_addrs=127.9.44.126:37605,127.9.44.125:39361,127.9.44.124:41697 with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:36.580031 20348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:36.580298 20348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:36.580431 20348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:36.587772 20348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:36.587881 20348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:36.588032 20348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:27:36.597247 20348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:40499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.67.keytab
--principal=oryx/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:33679
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=35655
--tserver_master_addrs=127.9.44.126:37605,127.9.44.125:39361,127.9.44.124:41697
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:36.598444 20348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:36.599740 20348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:36.611977 20355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:36.611941 20359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:36.612144 20362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:36.612764 20348 server_base.cc:1034] running on GCE node
I20260808 23:27:36.613294 20348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:36.614118 20348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:36.615311 20348 hybrid_clock.cc:648] HybridClock initialized: now 1786231656615266 us; error 39 us; skew 500 ppm
I20260808 23:27:36.615585 20348 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:36 dist-test-slave-6bbx krb5kdc[18406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231656, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:36.623716 20348 init.cc:377] Logged in from keytab as oryx/127.9.44.67@KRBTEST.COM (short username oryx)
I20260808 23:27:36.625139 20348 webserver.cc:460] Webserver started at http://127.9.44.67:35655/ using document root <none> and password file <none>
I20260808 23:27:36.625766 20348 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:36.625890 20348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:36.630966 20348 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:27:36.633486 20370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:36.634683 20348 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 23:27:36.634843 20348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "518525b775c4450a8105feaa7126331b"
format_stamp: "Formatted at 2026-08-08 23:27:31 on dist-test-slave-6bbx"
server_key: "2a88675930efce1df8e5585717b6b18e"
server_key_iv: "1a21e4198870fca4a7b4ff1fdc818344"
server_key_version: "encryptionkey@0"
I20260808 23:27:36.635769 20348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:36.654335 20348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:36.657970 20348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:36.658195 20348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:36.659019 20348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:36.660311 20348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:36.660405 20348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:36.660506 20348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:36.660574 20348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:36.690404 20348 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:33679
I20260808 23:27:36.690469 20562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:33679 every 8 connection(s)
I20260808 23:27:36.691813 20348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:27:36.698448 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 20348
Aug 08 23:27:36 dist-test-slave-6bbx krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231656, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM
Aug 08 23:27:36 dist-test-slave-6bbx krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231656, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.125@KRBTEST.COM
Aug 08 23:27:36 dist-test-slave-6bbx krb5kdc[18406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231656, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.124@KRBTEST.COM
I20260808 23:27:36.723774 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.696946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:49229 (local address 127.9.44.125:39361)
0808 23:27:36.699984 (+ 3038us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:36.699989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:36.700006 (+ 17us) server_negotiation.cc:407] Connection header received
0808 23:27:36.700075 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:36.700080 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:36.700155 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:36.700259 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:36.702953 (+ 2694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.703696 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.707929 (+ 4233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.708564 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.708971 (+ 407us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.716298 (+ 7327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:36.716305 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:36.716309 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:36.716373 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:36.720022 (+ 3649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:36.721078 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:36.721084 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:36.721088 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:36.721239 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:36.721711 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:36.721717 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:36.721720 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:36.722067 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:36.722225 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:36.723041 (+ 816us) server_negotiation.cc:299] Negotiation successful
0808 23:27:36.723299 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2930,"thread_start_us":2805,"threads_started":1}
I20260808 23:27:36.726228 20573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.694566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:53417 (local address 127.9.44.126:37605)
0808 23:27:36.694972 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:36.694977 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:36.699366 (+ 4389us) server_negotiation.cc:407] Connection header received
0808 23:27:36.700484 (+ 1118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:36.700489 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:36.700567 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:36.700679 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:36.702672 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.703783 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.706885 (+ 3102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.707634 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.707779 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.715530 (+ 7751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:36.715537 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:36.715542 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:36.715579 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:36.719357 (+ 3778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:36.723010 (+ 3653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:36.723016 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:36.723019 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:36.723164 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:36.724395 (+ 1231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:36.724401 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:36.724405 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:36.724908 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:36.725040 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:36.725476 (+ 436us) server_negotiation.cc:299] Negotiation successful
0808 23:27:36.725696 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":223,"threads_started":1}
I20260808 23:27:36.726790 20581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.696606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:39361 (local address 127.9.44.67:49229)
0808 23:27:36.697136 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:36.697159 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:36.699037 (+ 1878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:36.700428 (+ 1391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:36.700440 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:36.701568 (+ 1128us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:36.702811 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:36.702824 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.703880 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.703885 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:36.707788 (+ 3903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:36.707805 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.709690 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.709697 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:36.709876 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.712503 (+ 2627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:36.712527 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:36.716140 (+ 3613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:36.720177 (+ 4037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:36.720186 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:36.720202 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:36.720918 (+ 716us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:36.721354 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:36.721358 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:36.721362 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:36.721596 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:36.722224 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:36.722233 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:36.722609 (+ 376us) client_negotiation.cc:769] Sending connection context
0808 23:27:36.722940 (+ 331us) client_negotiation.cc:240] Negotiation successful
0808 23:27:36.723880 (+ 940us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":49,"spinlock_wait_cycles":16256,"thread_start_us":472,"threads_started":1}
I20260808 23:27:36.728246 20570 heartbeater.cc:347] Connected to a master server at 127.9.44.125:39361
I20260808 23:27:36.728523 20570 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:36.729373 20570 heartbeater.cc:511] Master 127.9.44.125:39361 requested a full tablet report, sending...
I20260808 23:27:36.730983 20575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.695155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.9.44.67:53417)
0808 23:27:36.696039 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:36.696088 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:36.700186 (+ 4098us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:36.700695 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:36.700699 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:36.701846 (+ 1147us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:36.702493 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:36.702519 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.703911 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.703916 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:36.706742 (+ 2826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:36.706760 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.708053 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.708059 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:36.708297 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.710231 (+ 1934us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:36.710269 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:36.715290 (+ 5021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:36.722381 (+ 7091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:36.722385 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:36.722389 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:36.722906 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:36.723790 (+ 884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:36.723794 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:36.723798 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:36.724271 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:36.725044 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:36.725048 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:36.725195 (+ 147us) client_negotiation.cc:769] Sending connection context
0808 23:27:36.725352 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 23:27:36.725598 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"mutex_wait_us":44,"spinlock_wait_cycles":81280,"thread_start_us":663,"threads_started":1}
I20260808 23:27:36.732097 20563 heartbeater.cc:347] Connected to a master server at 127.9.44.126:37605
I20260808 23:27:36.732231 20563 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:36.732271 19426 ts_manager.cc:194] Registered new tserver with Master: 518525b775c4450a8105feaa7126331b (127.9.44.67:33679)
I20260808 23:27:36.732748 20563 heartbeater.cc:511] Master 127.9.44.126:37605 requested a full tablet report, sending...
I20260808 23:27:36.733770 18464 ts_manager.cc:194] Re-registered known tserver with Master: 518525b775c4450a8105feaa7126331b (127.9.44.67:33679)
I20260808 23:27:36.734948 18464 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.44.67@KRBTEST.COM'} at 127.9.44.67:53417
I20260808 23:27:36.738282 20574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.695072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:41697 (local address 127.9.44.67:49711)
0808 23:27:36.696040 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:36.696095 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:36.698836 (+ 2741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:36.700428 (+ 1592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:36.700440 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:36.701552 (+ 1112us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:36.702493 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:36.702519 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.716308 (+ 13789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.716313 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:36.717691 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:36.717707 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.718742 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.718750 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:36.718922 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.719447 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:36.719469 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:36.722183 (+ 2714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:36.730046 (+ 7863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:36.730051 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:36.730055 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:36.730488 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:36.731302 (+ 814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:36.731307 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:36.731310 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:36.731476 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:36.736162 (+ 4686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:36.736167 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:36.737376 (+ 1209us) client_negotiation.cc:769] Sending connection context
0808 23:27:36.737555 (+ 179us) client_negotiation.cc:240] Negotiation successful
0808 23:27:36.737858 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"mutex_wait_us":1112,"spinlock_wait_cycles":47232,"thread_start_us":357,"threads_started":1}
I20260808 23:27:36.738689 20580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.695717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:49711 (local address 127.9.44.124:41697)
0808 23:27:36.699802 (+ 4085us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:36.699807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:36.699827 (+ 20us) server_negotiation.cc:407] Connection header received
0808 23:27:36.699902 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:36.699907 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:36.699995 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:36.700193 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:36.702725 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.709489 (+ 6764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.717902 (+ 8413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.718532 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.719229 (+ 697us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.723231 (+ 4002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:36.723236 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:36.723241 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:36.723274 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:36.729885 (+ 6611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:36.730641 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:36.730646 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:36.730649 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:36.730804 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:36.731615 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:36.731619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:36.731622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:36.736005 (+ 4383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:36.736447 (+ 442us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:36.737678 (+ 1231us) server_negotiation.cc:299] Negotiation successful
0808 23:27:36.737918 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3966,"thread_start_us":3801,"threads_started":1}
I20260808 23:27:36.739471 20564 heartbeater.cc:347] Connected to a master server at 127.9.44.124:41697
I20260808 23:27:36.739605 20564 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:36.740132 20564 heartbeater.cc:511] Master 127.9.44.124:41697 requested a full tablet report, sending...
I20260808 23:27:36.741303 19773 ts_manager.cc:194] Registered new tserver with Master: 518525b775c4450a8105feaa7126331b (127.9.44.67:33679)
I20260808 23:27:36.741518 20590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.704289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42008 (local address 127.9.44.126:37605)
0808 23:27:36.710572 (+ 6283us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:36.710578 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:36.710596 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:27:36.710647 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:36.710651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:36.710718 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:36.710818 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:36.713295 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.714505 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.727867 (+ 13362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.728487 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.728617 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.733086 (+ 4469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:36.733092 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:36.733097 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:36.733139 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:36.736596 (+ 3457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:36.737954 (+ 1358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:36.737959 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:36.737962 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:36.738110 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:36.739176 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:36.739181 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:36.739185 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:36.739518 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:36.739767 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:36.740926 (+ 1159us) server_negotiation.cc:299] Negotiation successful
0808 23:27:36.741135 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3353,"thread_start_us":3217,"threads_started":1}
I20260808 23:27:36.749264 20589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.704132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45350 (local address 127.9.44.124:41697)
0808 23:27:36.734507 (+ 30375us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:36.734511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:36.734526 (+ 15us) server_negotiation.cc:407] Connection header received
0808 23:27:36.734571 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:36.734576 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:36.734641 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:36.734733 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:36.736408 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.737469 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.739298 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.739889 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.740000 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.743227 (+ 3227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:36.743232 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:36.743235 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:36.743264 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:36.746647 (+ 3383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:36.747217 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:36.747222 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:36.747226 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:36.747386 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:36.747810 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:36.747816 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:36.747820 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:36.748115 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:36.748348 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:36.748814 (+ 466us) server_negotiation.cc:299] Negotiation successful
0808 23:27:36.749004 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30266,"thread_start_us":30768,"threads_started":1}
I20260808 23:27:36.750417 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.710143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55282 (local address 127.9.44.125:39361)
0808 23:27:36.724138 (+ 13995us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:36.724143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:36.725999 (+ 1856us) server_negotiation.cc:407] Connection header received
0808 23:27:36.726235 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:36.726240 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:36.726333 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:36.726411 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:36.730377 (+ 3966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.731351 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.738277 (+ 6926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.738876 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.738973 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.743903 (+ 4930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:36.743908 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:36.743911 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:36.743940 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:36.746652 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:36.747746 (+ 1094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:36.747752 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:36.747755 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:36.747890 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:36.748281 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:36.748286 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:36.748288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:36.748626 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:36.748843 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:36.749933 (+ 1090us) server_negotiation.cc:299] Negotiation successful
0808 23:27:36.750135 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13831,"thread_start_us":22194,"threads_started":1}
I20260808 23:27:36.746019 18464 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42008:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:27:36.754515 18464 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:27:36.792050 20609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.770613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54376 (local address 127.9.44.67:33679)
0808 23:27:36.771274 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:36.771283 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:36.784269 (+ 12986us) server_negotiation.cc:407] Connection header received
0808 23:27:36.784520 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:36.784531 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:36.784812 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:36.785014 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:36.785561 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.787248 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.789500 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.791056 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.791182 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.791394 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:36.791544 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 23:27:36.791740 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":770,"threads_started":1}
I20260808 23:27:36.792276 20608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.770308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:33679 (local address 127.0.0.1:54376)
0808 23:27:36.784139 (+ 13831us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:36.784157 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:36.784323 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:36.785015 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:36.785019 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:36.785043 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:36.785344 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:36.785364 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.787383 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.787388 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:36.789338 (+ 1950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:36.789354 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.791249 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.791254 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:36.791373 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.791397 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 23:27:36.791478 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 23:27:36.791596 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13684,"thread_start_us":21318,"threads_started":1}
I20260808 23:27:36.798341 20607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.770649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:39169 (local address 127.0.0.1:40292)
0808 23:27:36.790662 (+ 20013us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:36.790680 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:36.790789 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:36.793034 (+ 2245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:36.793037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:36.793057 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:36.793358 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:36.793367 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.794544 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.794550 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:36.796063 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:36.796078 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.797554 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.797561 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:36.797752 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.797787 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 23:27:36.797876 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 23:27:36.797990 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19904,"thread_start_us":12923,"threads_started":1}
I20260808 23:27:36.799173 20610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.771581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40292 (local address 127.9.44.66:39169)
0808 23:27:36.791246 (+ 19665us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:36.791255 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:36.791282 (+ 27us) server_negotiation.cc:407] Connection header received
0808 23:27:36.792613 (+ 1331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:36.792623 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:36.792850 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:36.793055 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:36.793507 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.794416 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.796478 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.797334 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.797478 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.797667 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:36.798700 (+ 1033us) server_negotiation.cc:299] Negotiation successful
0808 23:27:36.798869 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19513,"thread_start_us":19386,"threads_started":1}
I20260808 23:27:36.800434 20444 tablet_service.cc:1467] Processing CreateTablet for tablet bf8ac7a272604cdaac07f624b17f4c4e (DEFAULT_TABLE table=test-table [id=2e21b2245e13412bab99b6dabfe08136]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:27:36.802065 20444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf8ac7a272604cdaac07f624b17f4c4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:36.802949 20210 tablet_service.cc:1467] Processing CreateTablet for tablet bf8ac7a272604cdaac07f624b17f4c4e (DEFAULT_TABLE table=test-table [id=2e21b2245e13412bab99b6dabfe08136]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:27:36.804230 20210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf8ac7a272604cdaac07f624b17f4c4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:36.807340 20608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.792056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:42143 (local address 127.0.0.1:51734)
0808 23:27:36.792482 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:36.792498 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:36.792585 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:36.801132 (+ 8547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:36.801136 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:36.801161 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:36.801374 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:36.801389 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.803183 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.803188 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:36.804985 (+ 1797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:36.805000 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.806667 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.806674 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:36.806866 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.806901 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 23:27:36.806966 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 23:27:36.807090 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":4404,"threads_started":1}
I20260808 23:27:36.808522 20616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.792353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51734 (local address 127.9.44.65:42143)
0808 23:27:36.798089 (+ 5736us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:36.798097 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:36.798123 (+ 26us) server_negotiation.cc:407] Connection header received
0808 23:27:36.798224 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:36.798231 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:36.798436 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:36.801154 (+ 2718us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:36.801533 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.803034 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.805161 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.806498 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.807578 (+ 1080us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.808009 (+ 431us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:36.808085 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 23:27:36.808236 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5570,"thread_start_us":5455,"threads_started":1}
I20260808 23:27:36.811918 19986 tablet_service.cc:1467] Processing CreateTablet for tablet bf8ac7a272604cdaac07f624b17f4c4e (DEFAULT_TABLE table=test-table [id=2e21b2245e13412bab99b6dabfe08136]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:27:36.813299 19986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf8ac7a272604cdaac07f624b17f4c4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:36.814810 20620 tablet_bootstrap.cc:492] T bf8ac7a272604cdaac07f624b17f4c4e P 518525b775c4450a8105feaa7126331b: Bootstrap starting.
I20260808 23:27:36.817699 20620 tablet_bootstrap.cc:654] T bf8ac7a272604cdaac07f624b17f4c4e P 518525b775c4450a8105feaa7126331b: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:36.818871 20621 tablet_bootstrap.cc:492] T bf8ac7a272604cdaac07f624b17f4c4e P a140e60268f342a29eff122f628bbab5: Bootstrap starting.
I20260808 23:27:36.818959 20620 log.cc:824] T bf8ac7a272604cdaac07f624b17f4c4e P 518525b775c4450a8105feaa7126331b: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:36.822528 20621 tablet_bootstrap.cc:654] T bf8ac7a272604cdaac07f624b17f4c4e P a140e60268f342a29eff122f628bbab5: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:36.823980 20621 log.cc:824] T bf8ac7a272604cdaac07f624b17f4c4e P a140e60268f342a29eff122f628bbab5: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:36.830281 20620 tablet_bootstrap.cc:492] T bf8ac7a272604cdaac07f624b17f4c4e P 518525b775c4450a8105feaa7126331b: No bootstrap required, opened a new log
I20260808 23:27:36.830557 20620 ts_tablet_manager.cc:1397] T bf8ac7a272604cdaac07f624b17f4c4e P 518525b775c4450a8105feaa7126331b: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.004s
I20260808 23:27:36.830869 20621 tablet_bootstrap.cc:492] T bf8ac7a272604cdaac07f624b17f4c4e P a140e60268f342a29eff122f628bbab5: No bootstrap required, opened a new log
I20260808 23:27:36.831162 20621 ts_tablet_manager.cc:1397] T bf8ac7a272604cdaac07f624b17f4c4e P a140e60268f342a29eff122f628bbab5: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20260808 23:27:36.831156 20628 tablet_bootstrap.cc:492] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2: Bootstrap starting.
I20260808 23:27:36.834568 20628 tablet_bootstrap.cc:654] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:36.835774 20628 log.cc:824] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:36.838610 20628 tablet_bootstrap.cc:492] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2: No bootstrap required, opened a new log
I20260808 23:27:36.838891 20628 ts_tablet_manager.cc:1397] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260808 23:27:36.839036 20621 raft_consensus.cc:348] T bf8ac7a272604cdaac07f624b17f4c4e P a140e60268f342a29eff122f628bbab5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518525b775c4450a8105feaa7126331b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33679 } } peers { permanent_uuid: "a140e60268f342a29eff122f628bbab5" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 39169 } } peers { permanent_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42143 } }
I20260808 23:27:36.839373 20621 raft_consensus.cc:374] T bf8ac7a272604cdaac07f624b17f4c4e P a140e60268f342a29eff122f628bbab5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:36.839478 20621 raft_consensus.cc:729] T bf8ac7a272604cdaac07f624b17f4c4e P a140e60268f342a29eff122f628bbab5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a140e60268f342a29eff122f628bbab5, State: Initialized, Role: FOLLOWER
I20260808 23:27:36.840075 20621 consensus_queue.cc:260] T bf8ac7a272604cdaac07f624b17f4c4e P a140e60268f342a29eff122f628bbab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518525b775c4450a8105feaa7126331b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33679 } } peers { permanent_uuid: "a140e60268f342a29eff122f628bbab5" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 39169 } } peers { permanent_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42143 } }
I20260808 23:27:36.840338 20620 raft_consensus.cc:348] T bf8ac7a272604cdaac07f624b17f4c4e P 518525b775c4450a8105feaa7126331b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518525b775c4450a8105feaa7126331b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33679 } } peers { permanent_uuid: "a140e60268f342a29eff122f628bbab5" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 39169 } } peers { permanent_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42143 } }
I20260808 23:27:36.840720 20620 raft_consensus.cc:374] T bf8ac7a272604cdaac07f624b17f4c4e P 518525b775c4450a8105feaa7126331b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:36.840812 20620 raft_consensus.cc:729] T bf8ac7a272604cdaac07f624b17f4c4e P 518525b775c4450a8105feaa7126331b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 518525b775c4450a8105feaa7126331b, State: Initialized, Role: FOLLOWER
I20260808 23:27:36.841368 20620 consensus_queue.cc:260] T bf8ac7a272604cdaac07f624b17f4c4e P 518525b775c4450a8105feaa7126331b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518525b775c4450a8105feaa7126331b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33679 } } peers { permanent_uuid: "a140e60268f342a29eff122f628bbab5" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 39169 } } peers { permanent_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42143 } }
I20260808 23:27:36.842480 20563 heartbeater.cc:503] Master 127.9.44.126:37605 was elected leader, sending a full tablet report...
I20260808 23:27:36.843057 20620 ts_tablet_manager.cc:1428] T bf8ac7a272604cdaac07f624b17f4c4e P 518525b775c4450a8105feaa7126331b: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
I20260808 23:27:36.848409 20324 heartbeater.cc:503] Master 127.9.44.126:37605 was elected leader, sending a full tablet report...
I20260808 23:27:36.848475 20628 raft_consensus.cc:348] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518525b775c4450a8105feaa7126331b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33679 } } peers { permanent_uuid: "a140e60268f342a29eff122f628bbab5" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 39169 } } peers { permanent_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42143 } }
I20260808 23:27:36.848881 20628 raft_consensus.cc:374] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:36.848980 20628 raft_consensus.cc:729] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1ac6dffcc5c43dcaaec28cb6b1130f2, State: Initialized, Role: FOLLOWER
I20260808 23:27:36.849411 20621 ts_tablet_manager.cc:1428] T bf8ac7a272604cdaac07f624b17f4c4e P a140e60268f342a29eff122f628bbab5: Time spent starting tablet: real 0.018s user 0.008s sys 0.009s
I20260808 23:27:36.849581 20628 consensus_queue.cc:260] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518525b775c4450a8105feaa7126331b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33679 } } peers { permanent_uuid: "a140e60268f342a29eff122f628bbab5" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 39169 } } peers { permanent_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42143 } }
I20260808 23:27:36.854298 20628 ts_tablet_manager.cc:1428] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2: Time spent starting tablet: real 0.015s user 0.008s sys 0.007s
W20260808 23:27:36.865154 20328 tablet.cc:2369] T bf8ac7a272604cdaac07f624b17f4c4e P a140e60268f342a29eff122f628bbab5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:27:36.908006 20637 raft_consensus.cc:482] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:27:36.908267 20637 raft_consensus.cc:504] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518525b775c4450a8105feaa7126331b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33679 } } peers { permanent_uuid: "a140e60268f342a29eff122f628bbab5" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 39169 } } peers { permanent_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42143 } }
I20260808 23:27:36.909785 20637 leader_election.cc:288] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 518525b775c4450a8105feaa7126331b (127.9.44.67:33679), a140e60268f342a29eff122f628bbab5 (127.9.44.66:39169)
I20260808 23:27:36.917809 20610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.910123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:44203 (local address 127.9.44.66:39169)
0808 23:27:36.910310 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:36.910316 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:36.911129 (+ 813us) server_negotiation.cc:407] Connection header received
0808 23:27:36.911292 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:36.911297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:36.911385 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:36.911473 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:36.912258 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.913138 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.915656 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.916900 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.917035 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.917164 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:36.917430 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 23:27:36.917562 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 23:27:36.917858 20640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.910003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:39169 (local address 127.9.44.65:44203)
0808 23:27:36.910983 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:36.911004 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:36.911162 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:36.911493 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:36.911498 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:36.911530 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:36.912087 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:36.912097 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.913268 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.913273 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:36.915479 (+ 2206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:36.915495 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.917107 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.917112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:36.917266 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.917296 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 23:27:36.917371 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 23:27:36.917458 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":824,"mutex_wait_us":39,"spinlock_wait_cycles":29568,"thread_start_us":694,"threads_started":1}
I20260808 23:27:36.918012 20609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.910281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:36725 (local address 127.9.44.67:33679)
0808 23:27:36.910458 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:36.910464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:36.911106 (+ 642us) server_negotiation.cc:407] Connection header received
0808 23:27:36.911296 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:36.911301 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:36.911375 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:36.911473 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:36.912324 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.913036 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.915653 (+ 2617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.916922 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.917035 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.917139 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:36.917643 (+ 504us) server_negotiation.cc:299] Negotiation successful
0808 23:27:36.917791 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 23:27:36.918305 20638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.910167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:33679 (local address 127.9.44.65:36725)
0808 23:27:36.910988 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:36.911003 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:36.911162 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:36.911493 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:36.911498 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:36.911532 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:36.912084 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:36.912097 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.913171 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.913176 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:36.915479 (+ 2303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:36.915495 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.917294 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.917299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:36.917451 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.917484 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 23:27:36.917579 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 23:27:36.917704 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"mutex_wait_us":49,"spinlock_wait_cycles":7552,"thread_start_us":382,"threads_started":1}
I20260808 23:27:36.918825 20246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf8ac7a272604cdaac07f624b17f4c4e" candidate_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a140e60268f342a29eff122f628bbab5" is_pre_election: true
I20260808 23:27:36.919292 20246 raft_consensus.cc:2454] T bf8ac7a272604cdaac07f624b17f4c4e P a140e60268f342a29eff122f628bbab5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1ac6dffcc5c43dcaaec28cb6b1130f2 in term 0.
I20260808 23:27:36.919205 20483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf8ac7a272604cdaac07f624b17f4c4e" candidate_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "518525b775c4450a8105feaa7126331b" is_pre_election: true
I20260808 23:27:36.919727 20483 raft_consensus.cc:2454] T bf8ac7a272604cdaac07f624b17f4c4e P 518525b775c4450a8105feaa7126331b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1ac6dffcc5c43dcaaec28cb6b1130f2 in term 0.
I20260808 23:27:36.919971 19904 leader_election.cc:302] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a140e60268f342a29eff122f628bbab5, f1ac6dffcc5c43dcaaec28cb6b1130f2; no voters:
I20260808 23:27:36.920363 20637 raft_consensus.cc:2781] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:27:36.920550 20637 raft_consensus.cc:482] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:27:36.920673 20637 raft_consensus.cc:3037] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:36.922855 20637 raft_consensus.cc:504] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518525b775c4450a8105feaa7126331b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33679 } } peers { permanent_uuid: "a140e60268f342a29eff122f628bbab5" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 39169 } } peers { permanent_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42143 } }
I20260808 23:27:36.923403 20637 leader_election.cc:288] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2 [CANDIDATE]: Term 1 election: Requested vote from peers 518525b775c4450a8105feaa7126331b (127.9.44.67:33679), a140e60268f342a29eff122f628bbab5 (127.9.44.66:39169)
I20260808 23:27:36.923820 20483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf8ac7a272604cdaac07f624b17f4c4e" candidate_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "518525b775c4450a8105feaa7126331b"
I20260808 23:27:36.923839 20246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf8ac7a272604cdaac07f624b17f4c4e" candidate_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a140e60268f342a29eff122f628bbab5"
I20260808 23:27:36.924052 20483 raft_consensus.cc:3037] T bf8ac7a272604cdaac07f624b17f4c4e P 518525b775c4450a8105feaa7126331b [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:36.924084 20246 raft_consensus.cc:3037] T bf8ac7a272604cdaac07f624b17f4c4e P a140e60268f342a29eff122f628bbab5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:36.926424 20246 raft_consensus.cc:2454] T bf8ac7a272604cdaac07f624b17f4c4e P a140e60268f342a29eff122f628bbab5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1ac6dffcc5c43dcaaec28cb6b1130f2 in term 1.
I20260808 23:27:36.926440 20483 raft_consensus.cc:2454] T bf8ac7a272604cdaac07f624b17f4c4e P 518525b775c4450a8105feaa7126331b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1ac6dffcc5c43dcaaec28cb6b1130f2 in term 1.
I20260808 23:27:36.926807 19902 leader_election.cc:302] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 518525b775c4450a8105feaa7126331b, f1ac6dffcc5c43dcaaec28cb6b1130f2; no voters:
I20260808 23:27:36.927098 20637 raft_consensus.cc:2781] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:27:36.927505 20637 raft_consensus.cc:686] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2 [term 1 LEADER]: Becoming Leader. State: Replica: f1ac6dffcc5c43dcaaec28cb6b1130f2, State: Running, Role: LEADER
I20260808 23:27:36.928220 20637 consensus_queue.cc:237] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518525b775c4450a8105feaa7126331b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33679 } } peers { permanent_uuid: "a140e60268f342a29eff122f628bbab5" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 39169 } } peers { permanent_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42143 } }
I20260808 23:27:36.932838 18466 catalog_manager.cc:5495] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to f1ac6dffcc5c43dcaaec28cb6b1130f2 (127.9.44.65). New cstate: current_term: 1 leader_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518525b775c4450a8105feaa7126331b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a140e60268f342a29eff122f628bbab5" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 39169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42143 } health_report { overall_health: HEALTHY } } }
W20260808 23:27:36.944773 20571 tablet.cc:2369] T bf8ac7a272604cdaac07f624b17f4c4e P 518525b775c4450a8105feaa7126331b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:27:36.967908 20616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:36.963212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51740 (local address 127.9.44.65:42143)
0808 23:27:36.963407 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:36.963413 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:36.963521 (+ 108us) server_negotiation.cc:407] Connection header received
0808 23:27:36.963689 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:36.963692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:36.963780 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:36.963887 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:27:36.964179 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.964870 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.966083 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:36.966479 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:36.966563 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:36.966926 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:27:36.967380 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:27:36.967479 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:36.967553 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 23:27:36.967612 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 23:27:36.974109 20093 tablet.cc:2369] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:27:36.975632 20246 raft_consensus.cc:1261] T bf8ac7a272604cdaac07f624b17f4c4e P a140e60268f342a29eff122f628bbab5 [term 1 FOLLOWER]: Refusing update from remote peer f1ac6dffcc5c43dcaaec28cb6b1130f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:27:36.975632 20483 raft_consensus.cc:1261] T bf8ac7a272604cdaac07f624b17f4c4e P 518525b775c4450a8105feaa7126331b [term 1 FOLLOWER]: Refusing update from remote peer f1ac6dffcc5c43dcaaec28cb6b1130f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:27:36.976751 20641 consensus_queue.cc:1035] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "518525b775c4450a8105feaa7126331b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:27:36.977083 20637 consensus_queue.cc:1035] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a140e60268f342a29eff122f628bbab5" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 39169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:27:37.008067 18466 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:27:37.008354 18466 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:27:37.014130 18466 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e1653d349ee64235a944e5c763fc4945: Sending DeleteTablet for 3 replicas of tablet bf8ac7a272604cdaac07f624b17f4c4e
I20260808 23:27:37.015420 20444 tablet_service.cc:1514] Processing DeleteTablet for tablet bf8ac7a272604cdaac07f624b17f4c4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:27:37 UTC) from {username='oryx', principal='oryx/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:54376
I20260808 23:27:37.015925 20210 tablet_service.cc:1514] Processing DeleteTablet for tablet bf8ac7a272604cdaac07f624b17f4c4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:27:37 UTC) from {username='oryx', principal='oryx/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:40292
I20260808 23:27:37.016746 20663 tablet_replica.cc:330] T bf8ac7a272604cdaac07f624b17f4c4e P a140e60268f342a29eff122f628bbab5: stopping tablet replica
I20260808 23:27:37.016968 19986 tablet_service.cc:1514] Processing DeleteTablet for tablet bf8ac7a272604cdaac07f624b17f4c4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:27:37 UTC) from {username='oryx', principal='oryx/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:51734
I20260808 23:27:37.017278 20663 raft_consensus.cc:2229] T bf8ac7a272604cdaac07f624b17f4c4e P a140e60268f342a29eff122f628bbab5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:27:37.017798 20663 raft_consensus.cc:2258] T bf8ac7a272604cdaac07f624b17f4c4e P a140e60268f342a29eff122f628bbab5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:27:37.019960 20663 ts_tablet_manager.cc:1905] T bf8ac7a272604cdaac07f624b17f4c4e P a140e60268f342a29eff122f628bbab5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:27:37.027227 20663 ts_tablet_manager.cc:1918] T bf8ac7a272604cdaac07f624b17f4c4e P a140e60268f342a29eff122f628bbab5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:27:37.027364 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 18428
I20260808 23:27:37.027540 20663 log.cc:1196] T bf8ac7a272604cdaac07f624b17f4c4e P a140e60268f342a29eff122f628bbab5: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/bf8ac7a272604cdaac07f624b17f4c4e
I20260808 23:27:37.028244 20663 ts_tablet_manager.cc:1939] T bf8ac7a272604cdaac07f624b17f4c4e P a140e60268f342a29eff122f628bbab5: Deleting consensus metadata
I20260808 23:27:37.028424 18518 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:37.030242 18450 catalog_manager.cc:4841] TS a140e60268f342a29eff122f628bbab5 (127.9.44.66:39169): tablet bf8ac7a272604cdaac07f624b17f4c4e (table test-table [id=2e21b2245e13412bab99b6dabfe08136]) successfully deleted
I20260808 23:27:37.035197 20665 tablet_replica.cc:330] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2: stopping tablet replica
I20260808 23:27:37.035633 20665 raft_consensus.cc:2229] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:27:37.036324 20665 raft_consensus.cc:2258] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:27:37.038731 20665 ts_tablet_manager.cc:1905] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:27:37.043876 20662 tablet_replica.cc:330] T bf8ac7a272604cdaac07f624b17f4c4e P 518525b775c4450a8105feaa7126331b: stopping tablet replica
I20260808 23:27:37.044281 20662 raft_consensus.cc:2229] T bf8ac7a272604cdaac07f624b17f4c4e P 518525b775c4450a8105feaa7126331b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:27:37.044785 20662 raft_consensus.cc:2258] T bf8ac7a272604cdaac07f624b17f4c4e P 518525b775c4450a8105feaa7126331b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:27:37.046689 20665 ts_tablet_manager.cc:1918] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:27:37.046868 20665 log.cc:1196] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-0/wal/wals/bf8ac7a272604cdaac07f624b17f4c4e
I20260808 23:27:37.047492 20665 ts_tablet_manager.cc:1939] T bf8ac7a272604cdaac07f624b17f4c4e P f1ac6dffcc5c43dcaaec28cb6b1130f2: Deleting consensus metadata
I20260808 23:27:37.052902 20662 ts_tablet_manager.cc:1905] T bf8ac7a272604cdaac07f624b17f4c4e P 518525b775c4450a8105feaa7126331b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:27:37.053948 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:37.020409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55294 (local address 127.9.44.125:39361)
0808 23:27:37.020573 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:37.020579 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:37.026578 (+ 5999us) server_negotiation.cc:407] Connection header received
0808 23:27:37.027691 (+ 1113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:37.027698 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:37.027790 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:37.027865 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:37.033034 (+ 5169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:37.034219 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:37.037953 (+ 3734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:37.038581 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:37.038697 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:37.046016 (+ 7319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:37.046024 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:37.046028 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:37.046089 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:37.049703 (+ 3614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:37.050919 (+ 1216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:37.050925 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:37.050928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:37.051102 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:37.052105 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:37.052111 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:37.052115 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:37.052465 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:37.052699 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:37.053338 (+ 639us) server_negotiation.cc:299] Negotiation successful
0808 23:27:37.053569 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 23:27:37.059948 20662 ts_tablet_manager.cc:1918] T bf8ac7a272604cdaac07f624b17f4c4e P 518525b775c4450a8105feaa7126331b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:27:37.060149 20662 log.cc:1196] T bf8ac7a272604cdaac07f624b17f4c4e P 518525b775c4450a8105feaa7126331b: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/bf8ac7a272604cdaac07f624b17f4c4e
I20260808 23:27:37.060711 20662 ts_tablet_manager.cc:1939] T bf8ac7a272604cdaac07f624b17f4c4e P 518525b775c4450a8105feaa7126331b: Deleting consensus metadata
I20260808 23:27:37.070726 20589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:37.026512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45362 (local address 127.9.44.124:41697)
0808 23:27:37.026680 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:37.026686 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:37.055131 (+ 28445us) server_negotiation.cc:407] Connection header received
0808 23:27:37.055319 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:37.055325 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:37.055408 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:37.055507 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:37.057608 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:37.058660 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:37.060563 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:37.061183 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:37.061308 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:37.064346 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:37.064352 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:37.064358 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:37.064395 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:37.067886 (+ 3491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:37.068681 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:37.068687 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:37.068691 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:37.068840 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:37.069246 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:37.069251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:37.069255 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:37.069596 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:37.069869 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:37.070162 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 23:27:37.070393 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 23:27:37.175832 18448 catalog_manager.cc:4841] TS 518525b775c4450a8105feaa7126331b (127.9.44.67:33679): tablet bf8ac7a272604cdaac07f624b17f4c4e (table test-table [id=2e21b2245e13412bab99b6dabfe08136]) successfully deleted
I20260808 23:27:37.176163 18448 catalog_manager.cc:4841] TS f1ac6dffcc5c43dcaaec28cb6b1130f2 (127.9.44.65:42143): tablet bf8ac7a272604cdaac07f624b17f4c4e (table test-table [id=2e21b2245e13412bab99b6dabfe08136]) successfully deleted
I20260808 23:27:37.176446 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 18428
I20260808 23:27:37.223886 20589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:37.207367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45372 (local address 127.9.44.124:41697)
0808 23:27:37.207575 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:37.207583 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:37.209202 (+ 1619us) server_negotiation.cc:407] Connection header received
0808 23:27:37.209491 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:37.209496 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:37.209596 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:37.209698 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:37.211634 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:37.212397 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:37.213723 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:37.214403 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:37.214523 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:37.217436 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:37.217443 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:37.217447 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:37.217486 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:37.220884 (+ 3398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:37.221619 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:37.221633 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:37.221636 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:37.221795 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:37.222222 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:37.222227 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:37.222231 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:37.222598 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:37.222856 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:37.223410 (+ 554us) server_negotiation.cc:299] Negotiation successful
0808 23:27:37.223570 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 23:27:37.226922 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:37.208349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55310 (local address 127.9.44.125:39361)
0808 23:27:37.208550 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:37.208556 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:37.209359 (+ 803us) server_negotiation.cc:407] Connection header received
0808 23:27:37.209604 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:37.209609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:37.209683 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:37.209765 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:37.213622 (+ 3857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:37.214353 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:37.216286 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:37.216970 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:37.217094 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:37.219799 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:37.219804 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:37.219807 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:37.219857 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:37.222539 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:37.223108 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:37.223114 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:37.223117 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:37.223239 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:37.225584 (+ 2345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:37.225588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:37.225590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:37.225890 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:37.226114 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:37.226432 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 23:27:37.226640 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 23:27:37.998214 20683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:37.997460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.9.44.66:55079)
0808 23:27:37.997909 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:37.998050 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":243,"threads_started":1}
W20260808 23:27:37.998664 20324 heartbeater.cc:644] Failed to heartbeat to 127.9.44.126:37605 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
I20260808 23:27:38.003049 20684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.002263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.9.44.65:56843)
0808 23:27:38.002720 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.002866 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":223,"threads_started":1}
W20260808 23:27:38.003609 20082 heartbeater.cc:644] Failed to heartbeat to 127.9.44.126:37605 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
I20260808 23:27:38.007046 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.006382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.9.44.67:54081)
0808 23:27:38.006766 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.006877 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":181,"threads_started":1}
W20260808 23:27:38.007385 20563 heartbeater.cc:644] Failed to heartbeat to 127.9.44.126:37605 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
I20260808 23:27:38.514767 20686 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1653d349ee64235a944e5c763fc4945)
I20260808 23:27:38.515038 20686 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: false } }
I20260808 23:27:38.516429 20686 leader_election.cc:288] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1653d349ee64235a944e5c763fc4945 (127.9.44.126:37605), 8ab17fb16fb04158b064aee7a9998407 (127.9.44.124:41697)
I20260808 23:27:38.517598 20687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.516519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.0.0.1:46886)
0808 23:27:38.517190 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.517386 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":303,"threads_started":1}
W20260808 23:27:38.517851 19400 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
I20260808 23:27:38.519174 20687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.518871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.0.0.1:46890)
0808 23:27:38.518995 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.519057 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 23:27:38.519356 19400 leader_election.cc:334] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e1653d349ee64235a944e5c763fc4945 (127.9.44.126:37605): Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
I20260808 23:27:38.519855 20702 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e1653d349ee64235a944e5c763fc4945)
I20260808 23:27:38.520009 20702 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: false } }
I20260808 23:27:38.520754 20702 leader_election.cc:288] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e1653d349ee64235a944e5c763fc4945 (127.9.44.126:37605), d6758c943e0b4c7b89118814fd429e93 (127.9.44.125:39361)
I20260808 23:27:38.521983 20703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.520991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.0.0.1:46892)
0808 23:27:38.521463 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.521663 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":246,"threads_started":1}
W20260808 23:27:38.522926 19747 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
I20260808 23:27:38.523603 20690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.516641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:41697 (local address 127.0.0.1:48812)
0808 23:27:38.517357 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.517374 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:38.517513 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:38.517866 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:38.517871 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:38.517897 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:38.518197 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:38.518218 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.519259 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.519264 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:38.520821 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:38.520835 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.522771 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.522777 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:38.522961 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:38.523006 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 23:27:38.523123 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 23:27:38.523259 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":461,"threads_started":1}
I20260808 23:27:38.523653 20694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.516914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48812 (local address 127.9.44.124:41697)
0808 23:27:38.517360 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:38.517367 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:38.517451 (+ 84us) server_negotiation.cc:407] Connection header received
0808 23:27:38.517639 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:38.517644 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:38.517733 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:38.517829 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:38.518399 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.519119 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.520984 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.522393 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.522530 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:38.522657 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:38.523160 (+ 503us) server_negotiation.cc:299] Negotiation successful
0808 23:27:38.523311 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":335,"threads_started":1}
I20260808 23:27:38.524422 19787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d6758c943e0b4c7b89118814fd429e93" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8ab17fb16fb04158b064aee7a9998407" is_pre_election: true
I20260808 23:27:38.524751 19787 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6758c943e0b4c7b89118814fd429e93 in term 1.
I20260808 23:27:38.525261 19399 leader_election.cc:302] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ab17fb16fb04158b064aee7a9998407, d6758c943e0b4c7b89118814fd429e93; no voters: e1653d349ee64235a944e5c763fc4945
I20260808 23:27:38.525493 20704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.524967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.0.0.1:46900)
0808 23:27:38.525129 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.525298 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 23:27:38.525666 19747 leader_election.cc:334] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e1653d349ee64235a944e5c763fc4945 (127.9.44.126:37605): Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
I20260808 23:27:38.525870 20686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 23:27:38.525990 20686 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e1653d349ee64235a944e5c763fc4945)
I20260808 23:27:38.526083 20686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [term 1 FOLLOWER]: Advancing to term 2
I20260808 23:27:38.528775 20686 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: false } }
I20260808 23:27:38.529323 20703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.521617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:39361 (local address 127.0.0.1:55864)
0808 23:27:38.522290 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.522310 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:38.522429 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:38.523476 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:38.523481 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:38.523531 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:38.523801 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:38.523814 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.525491 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.525496 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:38.527368 (+ 1872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:38.527385 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.528596 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.528602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:38.528787 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:38.528822 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 23:27:38.528926 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 23:27:38.529057 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":1015,"threads_started":1}
I20260808 23:27:38.529383 20707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.521696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55864 (local address 127.9.44.125:39361)
0808 23:27:38.523175 (+ 1479us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:38.523180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:38.523200 (+ 20us) server_negotiation.cc:407] Connection header received
0808 23:27:38.523250 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:38.523255 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:38.523316 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:38.523960 (+ 644us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:38.524229 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.525321 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.527531 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.528418 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.528517 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:38.528596 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:38.528944 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 23:27:38.529052 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1371,"thread_start_us":1224,"threads_started":1}
I20260808 23:27:38.529551 20686 leader_election.cc:288] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [CANDIDATE]: Term 2 election: Requested vote from peers e1653d349ee64235a944e5c763fc4945 (127.9.44.126:37605), 8ab17fb16fb04158b064aee7a9998407 (127.9.44.124:41697)
I20260808 23:27:38.529963 20690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.529579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.0.0.1:46902)
0808 23:27:38.529751 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.529826 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260808 23:27:38.529978 19787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d6758c943e0b4c7b89118814fd429e93" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8ab17fb16fb04158b064aee7a9998407"
I20260808 23:27:38.530174 19787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [term 1 FOLLOWER]: Advancing to term 2
I20260808 23:27:38.530200 19444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8ab17fb16fb04158b064aee7a9998407" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d6758c943e0b4c7b89118814fd429e93" is_pre_election: true
I20260808 23:27:38.530586 19444 raft_consensus.cc:2379] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8ab17fb16fb04158b064aee7a9998407 in current term 2: Already voted for candidate d6758c943e0b4c7b89118814fd429e93 in this term.
I20260808 23:27:38.531009 20690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.530674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.0.0.1:46916)
0808 23:27:38.530797 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.530885 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 23:27:38.531124 19747 leader_election.cc:302] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ab17fb16fb04158b064aee7a9998407; no voters: d6758c943e0b4c7b89118814fd429e93, e1653d349ee64235a944e5c763fc4945
W20260808 23:27:38.531314 19400 leader_election.cc:334] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e1653d349ee64235a944e5c763fc4945 (127.9.44.126:37605): Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
I20260808 23:27:38.532727 19787 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6758c943e0b4c7b89118814fd429e93 in term 2.
I20260808 23:27:38.532896 20702 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 23:27:38.533094 19399 leader_election.cc:302] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ab17fb16fb04158b064aee7a9998407, d6758c943e0b4c7b89118814fd429e93; no voters: e1653d349ee64235a944e5c763fc4945
I20260808 23:27:38.533362 20686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 23:27:38.533731 20686 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [term 2 LEADER]: Becoming Leader. State: Replica: d6758c943e0b4c7b89118814fd429e93, State: Running, Role: LEADER
I20260808 23:27:38.534076 20686 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: false } }
I20260808 23:27:38.535868 20710 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6758c943e0b4c7b89118814fd429e93. Latest consensus state: current_term: 2 leader_uuid: "d6758c943e0b4c7b89118814fd429e93" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: false } } }
I20260808 23:27:38.536249 20710 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:38.536849 20712 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:27:38.541182 20712 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:27:38.542011 20712 catalog_manager.cc:1259] Loaded cluster ID: 072e9d2097594b749e6fc0670c92e369
I20260808 23:27:38.542080 20712 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:27:38.543162 20712 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:27:38.543987 20712 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93: Loaded TSK: 0
I20260808 23:27:38.544818 20712 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:27:38.596769 19424 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55310:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:27:38.603776 19424 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:27:38.607805 20690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.607394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.0.0.1:46944)
0808 23:27:38.607547 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.607618 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 23:27:38.607838 19787 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [term 2 FOLLOWER]: Refusing update from remote peer d6758c943e0b4c7b89118814fd429e93: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260808 23:27:38.608516 20710 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260808 23:27:38.608770 20690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.608382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.0.0.1:46950)
0808 23:27:38.608570 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.608635 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260808 23:27:38.608975 19400 consensus_peers.cc:487] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 -> Peer e1653d349ee64235a944e5c763fc4945 (127.9.44.126:37605): Couldn't send request to peer e1653d349ee64235a944e5c763fc4945. Status: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 23:27:38.611184 20702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6758c943e0b4c7b89118814fd429e93. Latest consensus state: current_term: 2 leader_uuid: "d6758c943e0b4c7b89118814fd429e93" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: false } } }
I20260808 23:27:38.611413 20702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:27:38.612884 20710 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "d6758c943e0b4c7b89118814fd429e93" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: false } } }
I20260808 23:27:38.613165 20710 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:38.614064 20715 mvcc.cc:205] Tried to move back new op lower bound from 7316404873651027968 to 7316404873358815232. Current Snapshot: MvccSnapshot[applied={T|T < 7316404873651027968}]
I20260808 23:27:38.614248 20702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "d6758c943e0b4c7b89118814fd429e93" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "e1653d349ee64235a944e5c763fc4945" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37605 } } peers { permanent_uuid: "d6758c943e0b4c7b89118814fd429e93" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 39361 } attrs { promote: false } } peers { permanent_uuid: "8ab17fb16fb04158b064aee7a9998407" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 41697 } attrs { promote: false } } }
I20260808 23:27:38.614507 20702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ab17fb16fb04158b064aee7a9998407 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:27:38.638185 20690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.620246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:39169 (local address 127.0.0.1:58268)
0808 23:27:38.620396 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.620414 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:38.620523 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:38.632276 (+ 11753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:38.632281 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:38.632307 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:38.632473 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:38.632485 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.633854 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.633860 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:38.635770 (+ 1910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:38.635785 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.637263 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.637269 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:38.637638 (+ 369us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:38.637671 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 23:27:38.637781 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 23:27:38.637924 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":143}
I20260808 23:27:38.638255 20724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.620377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58268 (local address 127.9.44.66:39169)
0808 23:27:38.632015 (+ 11638us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:38.632019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:38.632034 (+ 15us) server_negotiation.cc:407] Connection header received
0808 23:27:38.632091 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:38.632095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:38.632148 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:38.632237 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:38.632652 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.633709 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.635923 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.637057 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.637518 (+ 461us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:38.637613 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:38.637812 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 23:27:38.637951 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11541,"thread_start_us":11453,"threads_started":1}
I20260808 23:27:38.639057 20210 tablet_service.cc:1467] Processing CreateTablet for tablet 3924691268374d4d98735458212f1826 (DEFAULT_TABLE table=test-table [id=37bb4e98380c407d8b6d5d02251f5e89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:27:38.639562 20210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3924691268374d4d98735458212f1826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:38.640266 20687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.620655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:33679 (local address 127.0.0.1:60736)
0808 23:27:38.620832 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.620850 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:38.620943 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:38.634696 (+ 13753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:38.634700 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:38.634723 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:38.635002 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:38.635011 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.636083 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.636088 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:38.638084 (+ 1996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:38.638099 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.639436 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.639443 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:38.639609 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:38.639641 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 23:27:38.639758 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 23:27:38.639892 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 23:27:38.640628 20726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.621026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:42143 (local address 127.0.0.1:49190)
0808 23:27:38.621512 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.621528 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:38.622041 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:38.633944 (+ 11903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:38.633948 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:38.633982 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:38.634433 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:38.634446 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.635806 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.635812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:38.637480 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:38.637496 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.639776 (+ 2280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.639782 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:38.639957 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:38.639990 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 23:27:38.640081 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 23:27:38.640191 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"mutex_wait_us":55,"thread_start_us":782,"threads_started":1}
I20260808 23:27:38.640776 20729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.621374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49190 (local address 127.9.44.65:42143)
0808 23:27:38.633638 (+ 12264us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:38.633645 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:38.633664 (+ 19us) server_negotiation.cc:407] Connection header received
0808 23:27:38.633733 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:38.633738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:38.633814 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:38.633910 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:38.634609 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.635642 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.638231 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.639533 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.639701 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:38.639801 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:38.640134 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 23:27:38.640276 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12148,"thread_start_us":12061,"threads_started":1}
I20260808 23:27:38.641242 20727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.621374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60736 (local address 127.9.44.67:33679)
0808 23:27:38.634387 (+ 13013us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:38.634393 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:38.634412 (+ 19us) server_negotiation.cc:407] Connection header received
0808 23:27:38.634482 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:38.634487 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:38.634576 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:38.634665 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:38.635139 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.635951 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.638278 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.639246 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.639371 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:38.639473 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:38.640905 (+ 1432us) server_negotiation.cc:299] Negotiation successful
0808 23:27:38.641021 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12882,"thread_start_us":12777,"threads_started":1}
I20260808 23:27:38.641937 19986 tablet_service.cc:1467] Processing CreateTablet for tablet 3924691268374d4d98735458212f1826 (DEFAULT_TABLE table=test-table [id=37bb4e98380c407d8b6d5d02251f5e89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:27:38.641937 20444 tablet_service.cc:1467] Processing CreateTablet for tablet 3924691268374d4d98735458212f1826 (DEFAULT_TABLE table=test-table [id=37bb4e98380c407d8b6d5d02251f5e89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:27:38.642385 19986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3924691268374d4d98735458212f1826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:38.642385 20444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3924691268374d4d98735458212f1826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:38.645262 20730 tablet_bootstrap.cc:492] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5: Bootstrap starting.
I20260808 23:27:38.646754 20730 tablet_bootstrap.cc:654] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:38.648627 20730 tablet_bootstrap.cc:492] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5: No bootstrap required, opened a new log
I20260808 23:27:38.648933 20730 ts_tablet_manager.cc:1397] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 23:27:38.649151 20734 tablet_bootstrap.cc:492] T 3924691268374d4d98735458212f1826 P f1ac6dffcc5c43dcaaec28cb6b1130f2: Bootstrap starting.
I20260808 23:27:38.649732 20730 raft_consensus.cc:348] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a140e60268f342a29eff122f628bbab5" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 39169 } } peers { permanent_uuid: "518525b775c4450a8105feaa7126331b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33679 } } peers { permanent_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42143 } }
I20260808 23:27:38.649969 20730 raft_consensus.cc:374] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:38.650087 20730 raft_consensus.cc:729] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a140e60268f342a29eff122f628bbab5, State: Initialized, Role: FOLLOWER
I20260808 23:27:38.650918 20734 tablet_bootstrap.cc:654] T 3924691268374d4d98735458212f1826 P f1ac6dffcc5c43dcaaec28cb6b1130f2: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:38.650851 20730 consensus_queue.cc:260] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a140e60268f342a29eff122f628bbab5" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 39169 } } peers { permanent_uuid: "518525b775c4450a8105feaa7126331b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33679 } } peers { permanent_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42143 } }
I20260808 23:27:38.651652 20730 ts_tablet_manager.cc:1428] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 23:27:38.652015 20733 tablet_bootstrap.cc:492] T 3924691268374d4d98735458212f1826 P 518525b775c4450a8105feaa7126331b: Bootstrap starting.
I20260808 23:27:38.653913 20733 tablet_bootstrap.cc:654] T 3924691268374d4d98735458212f1826 P 518525b775c4450a8105feaa7126331b: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:38.655408 20327 heartbeater.cc:503] Master 127.9.44.125:39361 was elected leader, sending a full tablet report...
I20260808 23:27:38.655725 20739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.651870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.9.44.66:50051)
0808 23:27:38.652878 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.653002 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":870,"thread_start_us":731,"threads_started":1}
I20260808 23:27:38.656025 20734 tablet_bootstrap.cc:492] T 3924691268374d4d98735458212f1826 P f1ac6dffcc5c43dcaaec28cb6b1130f2: No bootstrap required, opened a new log
I20260808 23:27:38.656061 20733 tablet_bootstrap.cc:492] T 3924691268374d4d98735458212f1826 P 518525b775c4450a8105feaa7126331b: No bootstrap required, opened a new log
I20260808 23:27:38.656172 20734 ts_tablet_manager.cc:1397] T 3924691268374d4d98735458212f1826 P f1ac6dffcc5c43dcaaec28cb6b1130f2: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260808 23:27:38.656174 20733 ts_tablet_manager.cc:1397] T 3924691268374d4d98735458212f1826 P 518525b775c4450a8105feaa7126331b: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 23:27:38.656917 20733 raft_consensus.cc:348] T 3924691268374d4d98735458212f1826 P 518525b775c4450a8105feaa7126331b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a140e60268f342a29eff122f628bbab5" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 39169 } } peers { permanent_uuid: "518525b775c4450a8105feaa7126331b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33679 } } peers { permanent_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42143 } }
I20260808 23:27:38.656925 20734 raft_consensus.cc:348] T 3924691268374d4d98735458212f1826 P f1ac6dffcc5c43dcaaec28cb6b1130f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a140e60268f342a29eff122f628bbab5" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 39169 } } peers { permanent_uuid: "518525b775c4450a8105feaa7126331b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33679 } } peers { permanent_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42143 } }
I20260808 23:27:38.657148 20734 raft_consensus.cc:374] T 3924691268374d4d98735458212f1826 P f1ac6dffcc5c43dcaaec28cb6b1130f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:38.657148 20733 raft_consensus.cc:374] T 3924691268374d4d98735458212f1826 P 518525b775c4450a8105feaa7126331b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:38.657239 20733 raft_consensus.cc:729] T 3924691268374d4d98735458212f1826 P 518525b775c4450a8105feaa7126331b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 518525b775c4450a8105feaa7126331b, State: Initialized, Role: FOLLOWER
I20260808 23:27:38.657239 20734 raft_consensus.cc:729] T 3924691268374d4d98735458212f1826 P f1ac6dffcc5c43dcaaec28cb6b1130f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1ac6dffcc5c43dcaaec28cb6b1130f2, State: Initialized, Role: FOLLOWER
I20260808 23:27:38.657450 20734 consensus_queue.cc:260] T 3924691268374d4d98735458212f1826 P f1ac6dffcc5c43dcaaec28cb6b1130f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a140e60268f342a29eff122f628bbab5" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 39169 } } peers { permanent_uuid: "518525b775c4450a8105feaa7126331b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33679 } } peers { permanent_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42143 } }
I20260808 23:27:38.657473 20733 consensus_queue.cc:260] T 3924691268374d4d98735458212f1826 P 518525b775c4450a8105feaa7126331b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a140e60268f342a29eff122f628bbab5" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 39169 } } peers { permanent_uuid: "518525b775c4450a8105feaa7126331b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33679 } } peers { permanent_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42143 } }
I20260808 23:27:38.660099 20739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.656371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.9.44.66:36971)
0808 23:27:38.659867 (+ 3496us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.659952 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3400,"thread_start_us":3252,"threads_started":1}
I20260808 23:27:38.661706 20734 ts_tablet_manager.cc:1428] T 3924691268374d4d98735458212f1826 P f1ac6dffcc5c43dcaaec28cb6b1130f2: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 23:27:38.662658 20748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.661725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.9.44.65:32923)
0808 23:27:38.662315 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.662446 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":343,"threads_started":1}
I20260808 23:27:38.662786 20733 ts_tablet_manager.cc:1428] T 3924691268374d4d98735458212f1826 P 518525b775c4450a8105feaa7126331b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 23:27:38.663882 20088 heartbeater.cc:503] Master 127.9.44.125:39361 was elected leader, sending a full tablet report...
I20260808 23:27:38.664366 20748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.664034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.9.44.65:57249)
0808 23:27:38.664160 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.664237 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 23:27:38.664028 20749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.663020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.9.44.67:55371)
0808 23:27:38.663553 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.663698 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":257,"threads_started":1}
I20260808 23:27:38.666220 20570 heartbeater.cc:503] Master 127.9.44.125:39361 was elected leader, sending a full tablet report...
I20260808 23:27:38.666925 20749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.666590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.9.44.67:57597)
0808 23:27:38.666722 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.666802 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 23:27:38.728164 20737 raft_consensus.cc:482] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:27:38.728338 20737 raft_consensus.cc:504] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a140e60268f342a29eff122f628bbab5" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 39169 } } peers { permanent_uuid: "518525b775c4450a8105feaa7126331b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33679 } } peers { permanent_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42143 } }
I20260808 23:27:38.729787 20737 leader_election.cc:288] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 518525b775c4450a8105feaa7126331b (127.9.44.67:33679), f1ac6dffcc5c43dcaaec28cb6b1130f2 (127.9.44.65:42143)
I20260808 23:27:38.737066 20727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.730062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:58237 (local address 127.9.44.67:33679)
0808 23:27:38.730236 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:38.730240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:38.730380 (+ 140us) server_negotiation.cc:407] Connection header received
0808 23:27:38.730622 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:38.730628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:38.730705 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:38.730798 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:38.731251 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.732284 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.734722 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.735980 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.736103 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:38.736203 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:38.736696 (+ 493us) server_negotiation.cc:299] Negotiation successful
0808 23:27:38.736825 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 23:27:38.737075 20729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.730589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:60741 (local address 127.9.44.65:42143)
0808 23:27:38.730802 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:38.730808 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:38.730830 (+ 22us) server_negotiation.cc:407] Connection header received
0808 23:27:38.730895 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:38.730899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:38.730968 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:38.731035 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:38.731577 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.732525 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.734746 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.735986 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.736103 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:38.736203 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:38.736696 (+ 493us) server_negotiation.cc:299] Negotiation successful
0808 23:27:38.736825 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 23:27:38.737159 20742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.730473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:42143 (local address 127.9.44.66:60741)
0808 23:27:38.730658 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.730675 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:38.730786 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:38.731088 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:38.731092 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:38.731114 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:38.731425 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:38.731434 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.732648 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.732653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:38.734631 (+ 1978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:38.734642 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.736171 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.736179 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:38.736476 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:38.736570 (+ 94us) client_negotiation.cc:769] Sending connection context
0808 23:27:38.736663 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 23:27:38.736854 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":114}
I20260808 23:27:38.737181 20739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.730075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:33679 (local address 127.9.44.66:58237)
0808 23:27:38.730289 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.730308 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:38.730441 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:38.730828 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:38.730833 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:38.730858 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:38.731106 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:38.731117 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.732407 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.732411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:38.734553 (+ 2142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:38.734572 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.736208 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.736214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:38.736483 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:38.736572 (+ 89us) client_negotiation.cc:769] Sending connection context
0808 23:27:38.736663 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 23:27:38.736854 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":92}
I20260808 23:27:38.737932 20483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3924691268374d4d98735458212f1826" candidate_uuid: "a140e60268f342a29eff122f628bbab5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "518525b775c4450a8105feaa7126331b" is_pre_election: true
I20260808 23:27:38.738057 20015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3924691268374d4d98735458212f1826" candidate_uuid: "a140e60268f342a29eff122f628bbab5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" is_pre_election: true
I20260808 23:27:38.738198 20483 raft_consensus.cc:2454] T 3924691268374d4d98735458212f1826 P 518525b775c4450a8105feaa7126331b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a140e60268f342a29eff122f628bbab5 in term 0.
I20260808 23:27:38.738456 20015 raft_consensus.cc:2454] T 3924691268374d4d98735458212f1826 P f1ac6dffcc5c43dcaaec28cb6b1130f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a140e60268f342a29eff122f628bbab5 in term 0.
I20260808 23:27:38.738736 20134 leader_election.cc:302] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 518525b775c4450a8105feaa7126331b, a140e60268f342a29eff122f628bbab5; no voters:
I20260808 23:27:38.739104 20737 raft_consensus.cc:2781] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:27:38.739230 20737 raft_consensus.cc:482] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:27:38.739346 20737 raft_consensus.cc:3037] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:38.741619 20737 raft_consensus.cc:504] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a140e60268f342a29eff122f628bbab5" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 39169 } } peers { permanent_uuid: "518525b775c4450a8105feaa7126331b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33679 } } peers { permanent_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42143 } }
I20260808 23:27:38.742179 20737 leader_election.cc:288] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5 [CANDIDATE]: Term 1 election: Requested vote from peers 518525b775c4450a8105feaa7126331b (127.9.44.67:33679), f1ac6dffcc5c43dcaaec28cb6b1130f2 (127.9.44.65:42143)
I20260808 23:27:38.742511 20483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3924691268374d4d98735458212f1826" candidate_uuid: "a140e60268f342a29eff122f628bbab5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "518525b775c4450a8105feaa7126331b"
I20260808 23:27:38.742684 20483 raft_consensus.cc:3037] T 3924691268374d4d98735458212f1826 P 518525b775c4450a8105feaa7126331b [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:38.742625 20015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3924691268374d4d98735458212f1826" candidate_uuid: "a140e60268f342a29eff122f628bbab5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2"
I20260808 23:27:38.742803 20015 raft_consensus.cc:3037] T 3924691268374d4d98735458212f1826 P f1ac6dffcc5c43dcaaec28cb6b1130f2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:38.744706 20483 raft_consensus.cc:2454] T 3924691268374d4d98735458212f1826 P 518525b775c4450a8105feaa7126331b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a140e60268f342a29eff122f628bbab5 in term 1.
I20260808 23:27:38.744863 20015 raft_consensus.cc:2454] T 3924691268374d4d98735458212f1826 P f1ac6dffcc5c43dcaaec28cb6b1130f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a140e60268f342a29eff122f628bbab5 in term 1.
I20260808 23:27:38.745074 20134 leader_election.cc:302] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 518525b775c4450a8105feaa7126331b, a140e60268f342a29eff122f628bbab5; no voters:
I20260808 23:27:38.745347 20737 raft_consensus.cc:2781] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:27:38.745836 20737 raft_consensus.cc:686] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5 [term 1 LEADER]: Becoming Leader. State: Replica: a140e60268f342a29eff122f628bbab5, State: Running, Role: LEADER
I20260808 23:27:38.746243 20737 consensus_queue.cc:237] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a140e60268f342a29eff122f628bbab5" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 39169 } } peers { permanent_uuid: "518525b775c4450a8105feaa7126331b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33679 } } peers { permanent_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42143 } }
I20260808 23:27:38.746760 20742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.746355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.9.44.66:36781)
0808 23:27:38.746531 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.746617 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 23:27:38.750029 19424 catalog_manager.cc:5495] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5 reported cstate change: term changed from 0 to 1, leader changed from <none> to a140e60268f342a29eff122f628bbab5 (127.9.44.66). New cstate: current_term: 1 leader_uuid: "a140e60268f342a29eff122f628bbab5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a140e60268f342a29eff122f628bbab5" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 39169 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "518525b775c4450a8105feaa7126331b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42143 } health_report { overall_health: UNKNOWN } } }
I20260808 23:27:38.779860 20724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.775409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58274 (local address 127.9.44.66:39169)
0808 23:27:38.775566 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:38.775571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:38.775586 (+ 15us) server_negotiation.cc:407] Connection header received
0808 23:27:38.775697 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:38.775701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:38.775784 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:38.775891 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:27:38.776182 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.776834 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.777977 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.778425 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.778508 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:38.778900 (+ 392us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:27:38.779410 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:27:38.779517 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:38.779587 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 23:27:38.779645 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 23:27:38.783480 20483 raft_consensus.cc:1261] T 3924691268374d4d98735458212f1826 P 518525b775c4450a8105feaa7126331b [term 1 FOLLOWER]: Refusing update from remote peer a140e60268f342a29eff122f628bbab5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:27:38.783970 20015 raft_consensus.cc:1261] T 3924691268374d4d98735458212f1826 P f1ac6dffcc5c43dcaaec28cb6b1130f2 [term 1 FOLLOWER]: Refusing update from remote peer a140e60268f342a29eff122f628bbab5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:27:38.784188 20750 consensus_queue.cc:1035] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "518525b775c4450a8105feaa7126331b" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:27:38.784673 20737 consensus_queue.cc:1035] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1ac6dffcc5c43dcaaec28cb6b1130f2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:27:38.786504 20749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.786033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.9.44.67:44815)
0808 23:27:38.786199 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.786326 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 23:27:38.787712 20748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.787174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.9.44.65:40731)
0808 23:27:38.787402 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.787513 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260808 23:27:38.793931 20753 mvcc.cc:205] Tried to move back new op lower bound from 7316404874370760704 to 7316404874228797440. Current Snapshot: MvccSnapshot[applied={T|T < 7316404874370760704}]
I20260808 23:27:38.797763 20748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.796996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.9.44.65:56223)
0808 23:27:38.797555 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.797639 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457}
I20260808 23:27:38.801877 20749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.801403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.9.44.67:52225)
0808 23:27:38.801619 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.801703 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260808 23:27:38.803901 20742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.803430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.9.44.66:41917)
0808 23:27:38.803613 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.803743 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 23:27:38.807446 20742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.806761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.9.44.66:42509)
0808 23:27:38.807239 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.807318 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368}
I20260808 23:27:38.811942 19426 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:27:38.812227 19426 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:27:38.817687 19426 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d6758c943e0b4c7b89118814fd429e93: Sending DeleteTablet for 3 replicas of tablet 3924691268374d4d98735458212f1826
I20260808 23:27:38.818872 20210 tablet_service.cc:1514] Processing DeleteTablet for tablet 3924691268374d4d98735458212f1826 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:27:38 UTC) from {username='oryx', principal='oryx/127.9.44.125@KRBTEST.COM'} at 127.0.0.1:58268
I20260808 23:27:38.819164 19986 tablet_service.cc:1514] Processing DeleteTablet for tablet 3924691268374d4d98735458212f1826 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:27:38 UTC) from {username='oryx', principal='oryx/127.9.44.125@KRBTEST.COM'} at 127.0.0.1:49190
I20260808 23:27:38.819274 20444 tablet_service.cc:1514] Processing DeleteTablet for tablet 3924691268374d4d98735458212f1826 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:27:38 UTC) from {username='oryx', principal='oryx/127.9.44.125@KRBTEST.COM'} at 127.0.0.1:60736
I20260808 23:27:38.819690 20772 tablet_replica.cc:330] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5: stopping tablet replica
I20260808 23:27:38.819927 20772 raft_consensus.cc:2229] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:27:38.820618 20772 raft_consensus.cc:2258] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:27:38.821045 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 19722
I20260808 23:27:38.824235 20772 ts_tablet_manager.cc:1905] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:27:38.830886 20772 ts_tablet_manager.cc:1918] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:27:38.832506 20775 tablet_replica.cc:330] T 3924691268374d4d98735458212f1826 P 518525b775c4450a8105feaa7126331b: stopping tablet replica
I20260808 23:27:38.832727 20775 raft_consensus.cc:2229] T 3924691268374d4d98735458212f1826 P 518525b775c4450a8105feaa7126331b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:27:38.833026 20775 raft_consensus.cc:2258] T 3924691268374d4d98735458212f1826 P 518525b775c4450a8105feaa7126331b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:27:38.834086 20775 ts_tablet_manager.cc:1905] T 3924691268374d4d98735458212f1826 P 518525b775c4450a8105feaa7126331b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:27:38.834306 20772 log.cc:1196] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/3924691268374d4d98735458212f1826
I20260808 23:27:38.834852 20772 ts_tablet_manager.cc:1939] T 3924691268374d4d98735458212f1826 P a140e60268f342a29eff122f628bbab5: Deleting consensus metadata
I20260808 23:27:38.840467 19399 catalog_manager.cc:4841] TS a140e60268f342a29eff122f628bbab5 (127.9.44.66:39169): tablet 3924691268374d4d98735458212f1826 (table test-table [id=37bb4e98380c407d8b6d5d02251f5e89]) successfully deleted
I20260808 23:27:38.840832 20775 ts_tablet_manager.cc:1918] T 3924691268374d4d98735458212f1826 P 518525b775c4450a8105feaa7126331b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:27:38.840898 20773 tablet_replica.cc:330] T 3924691268374d4d98735458212f1826 P f1ac6dffcc5c43dcaaec28cb6b1130f2: stopping tablet replica
I20260808 23:27:38.840947 20775 log.cc:1196] T 3924691268374d4d98735458212f1826 P 518525b775c4450a8105feaa7126331b: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/3924691268374d4d98735458212f1826
I20260808 23:27:38.841095 20773 raft_consensus.cc:2229] T 3924691268374d4d98735458212f1826 P f1ac6dffcc5c43dcaaec28cb6b1130f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:27:38.841361 20775 ts_tablet_manager.cc:1939] T 3924691268374d4d98735458212f1826 P 518525b775c4450a8105feaa7126331b: Deleting consensus metadata
I20260808 23:27:38.841475 20773 raft_consensus.cc:2258] T 3924691268374d4d98735458212f1826 P f1ac6dffcc5c43dcaaec28cb6b1130f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:27:38.841854 20729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.821898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49192 (local address 127.9.44.65:42143)
0808 23:27:38.822065 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:38.822071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:38.823889 (+ 1818us) server_negotiation.cc:407] Connection header received
0808 23:27:38.824156 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:38.824161 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:38.824252 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:38.824331 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:38.826123 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.827424 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.829164 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.829805 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.829924 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:38.833397 (+ 3473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:38.833415 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:38.833434 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:38.833475 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:38.837527 (+ 4052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:38.838388 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:38.838397 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:38.838400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:38.838563 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:38.839118 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:38.839124 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:38.839128 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:38.839748 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:38.840127 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:38.841225 (+ 1098us) server_negotiation.cc:299] Negotiation successful
0808 23:27:38.841466 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 23:27:38.842406 19395 catalog_manager.cc:4841] TS 518525b775c4450a8105feaa7126331b (127.9.44.67:33679): tablet 3924691268374d4d98735458212f1826 (table test-table [id=37bb4e98380c407d8b6d5d02251f5e89]) successfully deleted
I20260808 23:27:38.842983 20773 ts_tablet_manager.cc:1905] T 3924691268374d4d98735458212f1826 P f1ac6dffcc5c43dcaaec28cb6b1130f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:27:38.843222 20076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:38.960217 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 19722
I20260808 23:27:38.986676 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 20108
I20260808 23:27:38.987195 20726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.986661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:42143 (local address 127.0.0.1:49202)
0808 23:27:38.986887 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:38.987015 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260808 23:27:38.987500 19395 catalog_manager.cc:4584] TS f1ac6dffcc5c43dcaaec28cb6b1130f2 (127.9.44.65:42143): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3924691268374d4d98735458212f1826: Network error: Client connection negotiation failed: client connection to 127.9.44.65:42143: connect: Connection refused (error 111)
I20260808 23:27:39.000895 20724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:38.987421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58286 (local address 127.9.44.66:39169)
0808 23:27:38.987589 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:38.987595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:38.987865 (+ 270us) server_negotiation.cc:407] Connection header received
0808 23:27:38.988115 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:38.988120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:38.988201 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:38.988288 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:38.989607 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.990435 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.991653 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:38.992278 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:38.992398 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:38.994925 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:38.994939 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:38.994955 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:38.995005 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:38.998143 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:38.998781 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:38.998788 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:38.998791 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:38.998974 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:38.999336 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:38.999342 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:38.999344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:38.999749 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:39.000096 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:39.000337 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 23:27:39.000551 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 23:27:39.001622 20319 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:39.047905 20726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:39.047466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:42143 (local address 127.0.0.1:49218)
0808 23:27:39.047649 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:39.047765 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 23:27:39.048847 20726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:39.048499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:42143 (local address 127.0.0.1:49220)
0808 23:27:39.048643 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:39.048717 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 23:27:39.100416 20726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:39.099908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.0.0.1:46956)
0808 23:27:39.100099 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:39.100218 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 23:27:39.101435 20726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:39.101040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37605 (local address 127.0.0.1:46958)
0808 23:27:39.101210 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:39.101298 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 23:27:39.117264 20726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:39.116861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:42143 (local address 127.0.0.1:49224)
0808 23:27:39.117022 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:39.117105 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 23:27:39.118151 20726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:39.117873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:42143 (local address 127.0.0.1:49240)
0808 23:27:39.117984 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:39.118047 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 23:27:39.120750 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 20108
I20260808 23:27:39.143489 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 20348
I20260808 23:27:39.156347 20727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:39.144200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60746 (local address 127.9.44.67:33679)
0808 23:27:39.144347 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:39.144352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:39.144398 (+ 46us) server_negotiation.cc:407] Connection header received
0808 23:27:39.144565 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:39.144569 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:39.144633 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:39.144718 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:39.146041 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:39.146919 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:39.148251 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:39.148636 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:39.148733 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:39.151098 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:39.151110 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:39.151127 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:39.151162 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:39.153796 (+ 2634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:39.154529 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:39.154536 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:39.154538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:39.154665 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:39.154984 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:39.154987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:39.154989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:39.155328 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:39.155548 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:39.155923 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 23:27:39.156086 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 23:27:39.156976 20556 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:39.219749 20726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:39.219169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:42143 (local address 127.0.0.1:49242)
0808 23:27:39.219411 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:39.219528 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260808 23:27:39.220777 20726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:39.220402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:42143 (local address 127.0.0.1:49252)
0808 23:27:39.220540 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:39.220623 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 23:27:39.247445 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 20348
I20260808 23:27:39.270447 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 19218
I20260808 23:27:39.271535 19474 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:39.391574 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 19218
W20260808 23:27:39.418509 19746 connection.cc:535] server connection from 127.0.0.1:48812 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 23:27:39.418895 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 19378
I20260808 23:27:39.419938 19827 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:39.533591 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 19378
2026-08-08T23:27:39Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11775 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:27:39 dist-test-slave-6bbx krb5kdc[20787](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:27:39 dist-test-slave-6bbx krb5kdc[20787](info): set up 2 sockets
Aug 08 23:27:39 dist-test-slave-6bbx krb5kdc[20787](info): commencing operation
krb5kdc: starting...
W20260808 23:27:40.738353 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:27:40 dist-test-slave-6bbx krb5kdc[20787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:27:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:27:40Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.126.keytab.
Entry for principal oryx/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.126.keytab.
I20260808 23:27:40.950848 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:43315
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.126.keytab
--principal=oryx/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:43315
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:41.277391 20811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:41.277647 20811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:41.277738 20811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:41.284773 20811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:27:41.284866 20811 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 23:27:41.284914 20811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:41.284971 20811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:41.285017 20811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:27:41.285075 20811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:27:41.294312 20811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:43315
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.126.keytab
--principal=oryx/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:43315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:41.295362 20811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:41.296651 20811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:41.307701 20821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:41.307595 20828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:41.307650 20822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:41.308511 20811 server_base.cc:1034] running on GCE node
I20260808 23:27:41.309185 20811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:41.310360 20811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:41.311578 20811 hybrid_clock.cc:648] HybridClock initialized: now 1786231661311512 us; error 39 us; skew 500 ppm
I20260808 23:27:41.311905 20811 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:41 dist-test-slave-6bbx krb5kdc[20787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231661, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:41.314731 20811 init.cc:377] Logged in from keytab as oryx/127.9.44.126@KRBTEST.COM (short username oryx)
I20260808 23:27:41.316102 20811 webserver.cc:460] Webserver started at http://127.9.44.126:34543/ using document root <none> and password file <none>
I20260808 23:27:41.316745 20811 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:41.316871 20811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:41.317179 20811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:41.319932 20811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "72837814baea4248ad80b660e43c5167"
format_stamp: "Formatted at 2026-08-08 23:27:41 on dist-test-slave-6bbx"
server_key: "a4fb3f21f3801043c2984f8a7a130f37"
server_key_iv: "c0add7e9352cb4354304adadf639e289"
server_key_version: "encryptionkey@0"
I20260808 23:27:41.320672 20811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "72837814baea4248ad80b660e43c5167"
format_stamp: "Formatted at 2026-08-08 23:27:41 on dist-test-slave-6bbx"
server_key: "a4fb3f21f3801043c2984f8a7a130f37"
server_key_iv: "c0add7e9352cb4354304adadf639e289"
server_key_version: "encryptionkey@0"
I20260808 23:27:41.325289 20811 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260808 23:27:41.328474 20840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:41.329658 20811 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 23:27:41.329824 20811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "72837814baea4248ad80b660e43c5167"
format_stamp: "Formatted at 2026-08-08 23:27:41 on dist-test-slave-6bbx"
server_key: "a4fb3f21f3801043c2984f8a7a130f37"
server_key_iv: "c0add7e9352cb4354304adadf639e289"
server_key_version: "encryptionkey@0"
I20260808 23:27:41.330024 20811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:41.357401 20811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:41.360970 20811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:41.361204 20811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:41.380791 20811 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:43315
I20260808 23:27:41.380846 20955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:43315 every 8 connection(s)
I20260808 23:27:41.382023 20811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:27:41.385627 20956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:41.388063 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 20811
I20260808 23:27:41.388253 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:27:41.388949 9393 external_mini_cluster.cc:1428] Setting key 8ed1150bd9aa3a69e8b265a05039251d
I20260808 23:27:41.397928 20956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167: Bootstrap starting.
I20260808 23:27:41.401378 20956 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:41.402324 20956 log.cc:824] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167: Log is configured to *not* fsync() on all Append() calls
Aug 08 23:27:41 dist-test-slave-6bbx krb5kdc[20787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM
I20260808 23:27:41.405459 20956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167: No bootstrap required, opened a new log
I20260808 23:27:41.412415 20960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:41.390839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58622 (local address 127.9.44.126:43315)
0808 23:27:41.391594 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:41.391608 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:41.391685 (+ 77us) server_negotiation.cc:407] Connection header received
0808 23:27:41.393817 (+ 2132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:41.393849 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:41.394365 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:41.394797 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:41.396052 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:41.397932 (+ 1880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:41.399195 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:41.400223 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:41.400395 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:41.404374 (+ 3979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:41.404386 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:41.404404 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:41.404444 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:41.408171 (+ 3727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:41.408793 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:41.408806 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:41.408814 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:41.409032 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:41.409348 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:41.409352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:41.409355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:41.410016 (+ 661us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:41.410280 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:41.410651 (+ 371us) server_negotiation.cc:299] Negotiation successful
0808 23:27:41.411080 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":307,"threads_started":1}
I20260808 23:27:41.415370 20956 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72837814baea4248ad80b660e43c5167" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43315 } }
I20260808 23:27:41.415690 20956 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:41.415776 20956 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72837814baea4248ad80b660e43c5167, State: Initialized, Role: FOLLOWER
I20260808 23:27:41.416250 20956 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72837814baea4248ad80b660e43c5167" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43315 } }
I20260808 23:27:41.416453 20956 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:27:41.416545 20956 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:27:41.416671 20956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:41.418612 20956 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72837814baea4248ad80b660e43c5167" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43315 } }
I20260808 23:27:41.419117 20956 leader_election.cc:302] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72837814baea4248ad80b660e43c5167; no voters:
I20260808 23:27:41.419714 20956 leader_election.cc:288] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:27:41.419843 20962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:27:41.420533 20962 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167 [term 1 LEADER]: Becoming Leader. State: Replica: 72837814baea4248ad80b660e43c5167, State: Running, Role: LEADER
I20260808 23:27:41.421066 20962 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72837814baea4248ad80b660e43c5167" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43315 } }
I20260808 23:27:41.421643 20956 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:27:41.423766 20968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72837814baea4248ad80b660e43c5167. Latest consensus state: current_term: 1 leader_uuid: "72837814baea4248ad80b660e43c5167" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72837814baea4248ad80b660e43c5167" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43315 } } }
I20260808 23:27:41.424222 20968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:41.424021 20965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72837814baea4248ad80b660e43c5167" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72837814baea4248ad80b660e43c5167" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43315 } } }
I20260808 23:27:41.424501 20965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:41.428380 20978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:27:41.432580 20978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:27:41.440618 20978 catalog_manager.cc:1347] Generated new cluster ID: 2d0694c5635b498481c0cbfd8a7a4862
I20260808 23:27:41.440725 20978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:27:41.447077 20978 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:27:41.448758 20978 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:27:41.457906 20978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167: Generated new TSK 0
I20260808 23:27:41.458722 20978 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.44.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.44.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.44.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.65.keytab.
Entry for principal oryx/127.9.44.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.65.keytab.
I20260808 23:27:41.514171 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:0
--local_ip_for_outbound_sockets=127.9.44.65
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:43315
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.65.keytab
--principal=oryx/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:41.831485 21000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:41.831838 21000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:41.831954 21000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:41.839157 21000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:41.839257 21000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:41.839344 21000 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 23:27:41.839421 21000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:27:41.848773 21000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.65.keytab
--principal=oryx/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.44.126:43315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:41.849872 21000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:41.851078 21000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:41.863443 21010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:41.863481 21012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:41.863519 21009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:41.868924 21000 server_base.cc:1034] running on GCE node
I20260808 23:27:41.869494 21000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:41.870322 21000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:41.871537 21000 hybrid_clock.cc:648] HybridClock initialized: now 1786231661871477 us; error 28 us; skew 500 ppm
I20260808 23:27:41.871837 21000 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:41 dist-test-slave-6bbx krb5kdc[20787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231661, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:41.878077 21000 init.cc:377] Logged in from keytab as oryx/127.9.44.65@KRBTEST.COM (short username oryx)
I20260808 23:27:41.879477 21000 webserver.cc:460] Webserver started at http://127.9.44.65:46443/ using document root <none> and password file <none>
I20260808 23:27:41.880169 21000 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:41.880256 21000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:41.880535 21000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:41.883318 21000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-0/data/instance:
uuid: "adec41244a7d4304aba0dcac828d454d"
format_stamp: "Formatted at 2026-08-08 23:27:41 on dist-test-slave-6bbx"
server_key: "b8a6a9e414c01a946a6c0fab694108e7"
server_key_iv: "d049ea86cdecfcccb62981badae33b28"
server_key_version: "encryptionkey@0"
I20260808 23:27:41.884088 21000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance:
uuid: "adec41244a7d4304aba0dcac828d454d"
format_stamp: "Formatted at 2026-08-08 23:27:41 on dist-test-slave-6bbx"
server_key: "b8a6a9e414c01a946a6c0fab694108e7"
server_key_iv: "d049ea86cdecfcccb62981badae33b28"
server_key_version: "encryptionkey@0"
I20260808 23:27:41.888744 21000 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 23:27:41.891976 21026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:41.893175 21000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:27:41.893322 21000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "adec41244a7d4304aba0dcac828d454d"
format_stamp: "Formatted at 2026-08-08 23:27:41 on dist-test-slave-6bbx"
server_key: "b8a6a9e414c01a946a6c0fab694108e7"
server_key_iv: "d049ea86cdecfcccb62981badae33b28"
server_key_version: "encryptionkey@0"
I20260808 23:27:41.893532 21000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:41.917143 21000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:41.920723 21000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:41.920967 21000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:41.923774 21000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:41.923887 21000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:41.924074 21000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:41.924183 21000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 23:27:41 dist-test-slave-6bbx krb5kdc[20787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231661, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM
I20260808 23:27:41.945221 21040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:41.924053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:47573 (local address 127.9.44.126:43315)
0808 23:27:41.924488 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:41.924494 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:41.925614 (+ 1120us) server_negotiation.cc:407] Connection header received
0808 23:27:41.927683 (+ 2069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:41.927690 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:41.927787 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:41.927901 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:41.929969 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:41.930930 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:41.933676 (+ 2746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:41.934340 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:41.934519 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:41.939243 (+ 4724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:41.939251 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:41.939256 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:41.939300 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:41.942153 (+ 2853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:41.942799 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:41.942805 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:41.942809 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:41.942923 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:41.943345 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:41.943351 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:41.943354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:41.943621 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:41.943804 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:41.944644 (+ 840us) server_negotiation.cc:299] Negotiation successful
0808 23:27:41.944875 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":218,"threads_started":1}
I20260808 23:27:41.946141 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:41.924725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:43315 (local address 127.9.44.65:47573)
0808 23:27:41.925467 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:41.925501 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:41.927383 (+ 1882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:41.928129 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:41.928152 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:41.928934 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:41.929788 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:41.929823 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:41.931149 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:41.931156 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:41.933492 (+ 2336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:41.933512 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:41.934567 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:41.934572 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:41.934778 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:41.935450 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:41.935489 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:41.938995 (+ 3506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:41.942285 (+ 3290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:41.942294 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:41.942308 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:41.942689 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:41.943027 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:41.943032 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:41.943036 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:41.943241 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:41.943781 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:41.943788 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:41.944217 (+ 429us) client_negotiation.cc:769] Sending connection context
0808 23:27:41.944558 (+ 341us) client_negotiation.cc:240] Negotiation successful
0808 23:27:41.945028 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":315,"threads_started":1}
I20260808 23:27:41.965144 21151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:41.952046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:43315 (local address 127.9.44.65:40029)
0808 23:27:41.952766 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:41.952787 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:41.952924 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:41.953212 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:41.953217 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:41.953821 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:41.954198 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:41.954211 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:41.955111 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:41.955116 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:41.956844 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:41.956861 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:41.957687 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:41.957693 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:41.957900 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:41.958523 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:41.958551 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:41.959543 (+ 992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:41.962254 (+ 2711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:41.962260 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:41.962264 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:41.962814 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:41.963152 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:41.963160 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:41.963164 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:41.963318 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:41.964020 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:41.964026 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:41.964229 (+ 203us) client_negotiation.cc:769] Sending connection context
0808 23:27:41.964408 (+ 179us) client_negotiation.cc:240] Negotiation successful
0808 23:27:41.964716 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":428,"threads_started":1}
I20260808 23:27:41.965456 21040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:41.952430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:40029 (local address 127.9.44.126:43315)
0808 23:27:41.952569 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:41.952574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:41.952869 (+ 295us) server_negotiation.cc:407] Connection header received
0808 23:27:41.953018 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:41.953022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:41.953077 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:41.953148 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:41.954327 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:41.955000 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:41.957002 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:41.957480 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:41.957622 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:41.959783 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:41.959789 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:41.959794 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:41.959831 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:41.962138 (+ 2307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:41.962916 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:41.962920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:41.962922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:41.963032 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:41.963489 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:41.963495 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:41.963498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:41.963886 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:41.964041 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:41.964863 (+ 822us) server_negotiation.cc:299] Negotiation successful
0808 23:27:41.965101 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 23:27:41.973263 21000 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:43811
I20260808 23:27:41.973306 21231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:43811 every 8 connection(s)
I20260808 23:27:41.974498 21000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:27:41.976274 21233 heartbeater.cc:347] Connected to a master server at 127.9.44.126:43315
I20260808 23:27:41.976469 21233 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:41.977028 21233 heartbeater.cc:511] Master 127.9.44.126:43315 requested a full tablet report, sending...
I20260808 23:27:41.978771 20870 ts_manager.cc:194] Registered new tserver with Master: adec41244a7d4304aba0dcac828d454d (127.9.44.65:43811)
I20260808 23:27:41.980181 20870 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.44.65@KRBTEST.COM'} at 127.9.44.65:47573
I20260808 23:27:41.984023 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 21000
I20260808 23:27:41.984196 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance
I20260808 23:27:41.984822 9393 external_mini_cluster.cc:1428] Setting key 928c83ce3eea30be40462581436b22cd
WARNING: no policy specified for oryx/127.9.44.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.44.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.44.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.66.keytab.
Entry for principal oryx/127.9.44.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.66.keytab.
I20260808 23:27:42.032485 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:0
--local_ip_for_outbound_sockets=127.9.44.66
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:43315
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.66.keytab
--principal=oryx/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:42.348444 21241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:42.348685 21241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:42.348809 21241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:42.355783 21241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:42.355887 21241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:42.356007 21241 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 23:27:42.356117 21241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:27:42.365568 21241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.66.keytab
--principal=oryx/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.44.126:43315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:42.366693 21241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:42.367946 21241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:42.379978 21248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:42.379971 21250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:42.380517 21254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:42.383898 21241 server_base.cc:1034] running on GCE node
I20260808 23:27:42.384549 21241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:42.385555 21241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:42.386795 21241 hybrid_clock.cc:648] HybridClock initialized: now 1786231662386714 us; error 48 us; skew 500 ppm
I20260808 23:27:42.387112 21241 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:42 dist-test-slave-6bbx krb5kdc[20787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231662, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:42.390756 21241 init.cc:377] Logged in from keytab as oryx/127.9.44.66@KRBTEST.COM (short username oryx)
I20260808 23:27:42.392438 21241 webserver.cc:460] Webserver started at http://127.9.44.66:39741/ using document root <none> and password file <none>
I20260808 23:27:42.393268 21241 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:42.393376 21241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:42.393687 21241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:42.397306 21241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-1/data/instance:
uuid: "d0341bb2dab042899a597a550f919011"
format_stamp: "Formatted at 2026-08-08 23:27:42 on dist-test-slave-6bbx"
server_key: "94738dd88b13959fda8035a3aefdbfa7"
server_key_iv: "fe6c20d36448938b6116ada49f6dcd21"
server_key_version: "encryptionkey@0"
I20260808 23:27:42.398258 21241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance:
uuid: "d0341bb2dab042899a597a550f919011"
format_stamp: "Formatted at 2026-08-08 23:27:42 on dist-test-slave-6bbx"
server_key: "94738dd88b13959fda8035a3aefdbfa7"
server_key_iv: "fe6c20d36448938b6116ada49f6dcd21"
server_key_version: "encryptionkey@0"
I20260808 23:27:42.403074 21241 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 23:27:42.406297 21266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:42.407452 21241 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 23:27:42.407572 21241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "d0341bb2dab042899a597a550f919011"
format_stamp: "Formatted at 2026-08-08 23:27:42 on dist-test-slave-6bbx"
server_key: "94738dd88b13959fda8035a3aefdbfa7"
server_key_iv: "fe6c20d36448938b6116ada49f6dcd21"
server_key_version: "encryptionkey@0"
I20260808 23:27:42.407819 21241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:42.462917 21241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:42.466653 21241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:42.466876 21241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:42.469395 21241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:42.469496 21241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:42.469672 21241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:42.469753 21241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 23:27:42 dist-test-slave-6bbx krb5kdc[20787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231662, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM
I20260808 23:27:42.490850 21284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:42.469892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:51005 (local address 127.9.44.126:43315)
0808 23:27:42.470652 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:42.470659 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:42.471615 (+ 956us) server_negotiation.cc:407] Connection header received
0808 23:27:42.473631 (+ 2016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:42.473636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:42.473727 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:42.473826 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:42.475868 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:42.476992 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:42.479279 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:42.479995 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:42.480136 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:42.484836 (+ 4700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:42.484843 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:42.484848 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:42.484890 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:42.487917 (+ 3027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:42.488619 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:42.488626 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:42.488630 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:42.488753 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:42.489181 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:42.489186 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:42.489190 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:42.489456 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:42.489588 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:42.490252 (+ 664us) server_negotiation.cc:299] Negotiation successful
0808 23:27:42.490483 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":530,"threads_started":1}
I20260808 23:27:42.491999 21285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:42.470584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:43315 (local address 127.9.44.66:51005)
0808 23:27:42.471443 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:42.471491 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:42.473377 (+ 1886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:42.474119 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:42.474131 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:42.474874 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:42.475633 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:42.475689 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:42.477172 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:42.477178 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:42.479122 (+ 1944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:42.479139 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:42.480222 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:42.480228 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:42.480475 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:42.481205 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:42.481239 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:42.484563 (+ 3324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:42.488053 (+ 3490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:42.488062 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:42.488077 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:42.488492 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:42.488856 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:42.488861 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:42.488865 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:42.489063 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:42.489582 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:42.489589 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:42.489903 (+ 314us) client_negotiation.cc:769] Sending connection context
0808 23:27:42.490125 (+ 222us) client_negotiation.cc:240] Negotiation successful
0808 23:27:42.490595 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":361,"threads_started":1}
I20260808 23:27:42.513083 21284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:42.499042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:50647 (local address 127.9.44.126:43315)
0808 23:27:42.499236 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:42.499241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:42.499721 (+ 480us) server_negotiation.cc:407] Connection header received
0808 23:27:42.499912 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:42.499917 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:42.499993 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:42.500089 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:42.501396 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:42.502319 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:42.504239 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:42.504870 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:42.504995 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:42.507201 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:42.507208 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:42.507213 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:42.507252 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:42.510348 (+ 3096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:42.511152 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:42.511158 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:42.511162 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:42.511318 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:42.511641 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:42.511646 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:42.511648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:42.512088 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:42.512259 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:42.512722 (+ 463us) server_negotiation.cc:299] Negotiation successful
0808 23:27:42.512880 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 23:27:42.513489 21407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:42.498970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:43315 (local address 127.9.44.66:50647)
0808 23:27:42.499592 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:42.499613 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:42.499790 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:42.500116 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:42.500121 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:42.500699 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:42.501190 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:42.501208 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:42.502482 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:42.502486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:42.504091 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:42.504105 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:42.505090 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:42.505097 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:42.505273 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:42.505924 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:42.505948 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:42.507000 (+ 1052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:42.510532 (+ 3532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:42.510537 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:42.510541 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:42.510989 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:42.511418 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:42.511421 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:42.511424 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:42.511535 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:42.512228 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:42.512233 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:42.512458 (+ 225us) client_negotiation.cc:769] Sending connection context
0808 23:27:42.512668 (+ 210us) client_negotiation.cc:240] Negotiation successful
0808 23:27:42.513002 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":319,"threads_started":1}
I20260808 23:27:42.518236 21241 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:45343
I20260808 23:27:42.518281 21472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:45343 every 8 connection(s)
I20260808 23:27:42.519343 21241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:27:42.521135 21473 heartbeater.cc:347] Connected to a master server at 127.9.44.126:43315
I20260808 23:27:42.521345 21473 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:42.521503 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 21241
I20260808 23:27:42.521672 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance
I20260808 23:27:42.522037 21473 heartbeater.cc:511] Master 127.9.44.126:43315 requested a full tablet report, sending...
I20260808 23:27:42.522099 9393 external_mini_cluster.cc:1428] Setting key be59a7f2a139bfb5f0aa1f8984d7958d
I20260808 23:27:42.523514 20868 ts_manager.cc:194] Registered new tserver with Master: d0341bb2dab042899a597a550f919011 (127.9.44.66:45343)
I20260808 23:27:42.524483 20868 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.44.66@KRBTEST.COM'} at 127.9.44.66:51005
WARNING: no policy specified for oryx/127.9.44.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.44.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.44.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.67.keytab.
Entry for principal oryx/127.9.44.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.67.keytab.
I20260808 23:27:42.570739 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:0
--local_ip_for_outbound_sockets=127.9.44.67
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:43315
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.67.keytab
--principal=oryx/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:42.885764 21477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:42.886091 21477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:42.886255 21477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:42.894675 21477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:42.894771 21477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:42.894889 21477 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 23:27:42.894991 21477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:27:42.904666 21477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.67.keytab
--principal=oryx/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.44.126:43315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:42.905978 21477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:42.907263 21477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:42.919709 21489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:42.919763 21494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:42.919811 21487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:42.920588 21477 server_base.cc:1034] running on GCE node
I20260808 23:27:42.921177 21477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:42.922013 21477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:42.923239 21477 hybrid_clock.cc:648] HybridClock initialized: now 1786231662923169 us; error 43 us; skew 500 ppm
I20260808 23:27:42.923513 21477 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:42 dist-test-slave-6bbx krb5kdc[20787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231662, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:42.926496 21477 init.cc:377] Logged in from keytab as oryx/127.9.44.67@KRBTEST.COM (short username oryx)
I20260808 23:27:42.927865 21477 webserver.cc:460] Webserver started at http://127.9.44.67:37343/ using document root <none> and password file <none>
I20260808 23:27:42.928520 21477 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:42.928609 21477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:42.928884 21477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:42.931587 21477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-2/data/instance:
uuid: "e3c4f5915fcc462194b8ee854a3bee72"
format_stamp: "Formatted at 2026-08-08 23:27:42 on dist-test-slave-6bbx"
server_key: "9423b8d36b9ccff017b817fa40607e8a"
server_key_iv: "e3318685cf88484459b9499ffdb89014"
server_key_version: "encryptionkey@0"
I20260808 23:27:42.932365 21477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance:
uuid: "e3c4f5915fcc462194b8ee854a3bee72"
format_stamp: "Formatted at 2026-08-08 23:27:42 on dist-test-slave-6bbx"
server_key: "9423b8d36b9ccff017b817fa40607e8a"
server_key_iv: "e3318685cf88484459b9499ffdb89014"
server_key_version: "encryptionkey@0"
I20260808 23:27:42.937273 21477 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:27:42.940516 21508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:42.941721 21477 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:27:42.941857 21477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "e3c4f5915fcc462194b8ee854a3bee72"
format_stamp: "Formatted at 2026-08-08 23:27:42 on dist-test-slave-6bbx"
server_key: "9423b8d36b9ccff017b817fa40607e8a"
server_key_iv: "e3318685cf88484459b9499ffdb89014"
server_key_version: "encryptionkey@0"
I20260808 23:27:42.942050 21477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:42.965340 21477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:42.968940 21477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:42.969185 21477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:42.971750 21477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:42.971860 21477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:42.971992 21477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:42.972049 21477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:42.982530 21233 heartbeater.cc:503] Master 127.9.44.126:43315 was elected leader, sending a full tablet report...
Aug 08 23:27:42 dist-test-slave-6bbx krb5kdc[20787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231662, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM
I20260808 23:27:42.995049 21284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:42.972083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:45263 (local address 127.9.44.126:43315)
0808 23:27:42.972315 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:42.972323 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:42.973680 (+ 1357us) server_negotiation.cc:407] Connection header received
0808 23:27:42.975811 (+ 2131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:42.975818 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:42.975937 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:42.976017 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:42.978358 (+ 2341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:42.979487 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:42.981992 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:42.982670 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:42.982810 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:42.987826 (+ 5016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:42.987833 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:42.987838 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:42.987882 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:42.991022 (+ 3140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:42.992099 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:42.992106 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:42.992110 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:42.992269 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:42.992898 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:42.992903 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:42.992907 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:42.993274 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:42.993444 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:42.994439 (+ 995us) server_negotiation.cc:299] Negotiation successful
0808 23:27:42.994668 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 23:27:42.996548 21520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:42.972668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:43315 (local address 127.9.44.67:45263)
0808 23:27:42.973525 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:42.973573 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:42.975455 (+ 1882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:42.976476 (+ 1021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:42.976487 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:42.977226 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:42.978105 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:42.978137 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:42.979640 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:42.979644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:42.981822 (+ 2178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:42.981834 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:42.982993 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:42.983002 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:42.983325 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:42.984169 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:42.984203 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:42.987532 (+ 3329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:42.991178 (+ 3646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:42.991191 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:42.991206 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:42.991920 (+ 714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:42.992424 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:42.992432 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:42.992436 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:42.992728 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:42.993439 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:42.993448 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:42.993928 (+ 480us) client_negotiation.cc:769] Sending connection context
0808 23:27:42.994288 (+ 360us) client_negotiation.cc:240] Negotiation successful
0808 23:27:42.994861 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":375,"threads_started":1}
I20260808 23:27:43.012585 21477 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:43007
I20260808 23:27:43.012835 21705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:43007 every 8 connection(s)
I20260808 23:27:43.014042 21477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:27:43.016410 21706 heartbeater.cc:347] Connected to a master server at 127.9.44.126:43315
I20260808 23:27:43.016594 21706 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:43.017236 21706 heartbeater.cc:511] Master 127.9.44.126:43315 requested a full tablet report, sending...
I20260808 23:27:43.017953 21284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:43.003429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:46475 (local address 127.9.44.126:43315)
0808 23:27:43.003617 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:43.003624 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:43.004209 (+ 585us) server_negotiation.cc:407] Connection header received
0808 23:27:43.004496 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:43.004502 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:43.004582 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:43.004688 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:43.006096 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.007197 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.009359 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.009969 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.010102 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:43.011870 (+ 1768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:43.011876 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:43.011880 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:43.011915 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:43.015156 (+ 3241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:43.015970 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:43.015975 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:43.015978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:43.016131 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:43.016488 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:43.016493 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:43.016497 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:43.016806 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:43.016948 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:43.017405 (+ 457us) server_negotiation.cc:299] Negotiation successful
0808 23:27:43.017599 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 23:27:43.018038 21671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:43.003335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:43315 (local address 127.9.44.67:46475)
0808 23:27:43.004103 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:43.004124 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:43.004336 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:43.004736 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:43.004741 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:43.005406 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:43.005907 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:43.005922 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.007398 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.007404 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:43.009198 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:43.009218 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.010135 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.010140 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:43.010245 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:43.010933 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:43.010949 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:43.011608 (+ 659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:43.015313 (+ 3705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:43.015319 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:43.015327 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:43.015829 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:43.016231 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:43.016235 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:43.016237 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:43.016365 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:43.016924 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:43.016929 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:43.017144 (+ 215us) client_negotiation.cc:769] Sending connection context
0808 23:27:43.017265 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 23:27:43.017567 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":16,"thread_start_us":370,"threads_started":1}
I20260808 23:27:43.018553 20868 ts_manager.cc:194] Registered new tserver with Master: e3c4f5915fcc462194b8ee854a3bee72 (127.9.44.67:43007)
I20260808 23:27:43.019500 20868 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.44.67@KRBTEST.COM'} at 127.9.44.67:45263
I20260808 23:27:43.019541 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 21477
I20260808 23:27:43.019719 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance
I20260808 23:27:43.020294 9393 external_mini_cluster.cc:1428] Setting key be0992f941b6e5da3d923dd06a4a54a0
I20260808 23:27:43.023240 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 23:27:43 dist-test-slave-6bbx krb5kdc[20787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231663, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 23:27:43 dist-test-slave-6bbx krb5kdc[20787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:43 dist-test-slave-6bbx krb5kdc[20787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 23:27:43.076109 21284 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:43.066459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58632 (local address 127.9.44.126:43315)
0808 23:27:43.066692 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:43.066698 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:43.066845 (+ 147us) server_negotiation.cc:407] Connection header received
0808 23:27:43.066996 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:43.067000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:43.067066 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:43.067151 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:43.068441 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.069172 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.070325 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.070724 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.070816 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:43.075809 (+ 4993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58632: BlockingRecv error: recv got EOF from 127.0.0.1:58632 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 08 23:27:43 dist-test-slave-6bbx krb5kdc[20787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231663, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM
I20260808 23:27:43.093142 21284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:43.079608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58640 (local address 127.9.44.126:43315)
0808 23:27:43.079781 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:43.079785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:43.080167 (+ 382us) server_negotiation.cc:407] Connection header received
0808 23:27:43.080367 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:43.080373 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:43.080449 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:43.080541 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:43.081981 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.083021 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.084663 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.085087 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.085182 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:43.088591 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:43.088597 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:43.088600 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:43.088631 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:43.091011 (+ 2380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:43.091584 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:43.091588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:43.091590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:43.091757 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:43.092054 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:43.092057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:43.092059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:43.092310 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:43.092521 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:43.092687 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 23:27:43.092838 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 23:27:43.097054 20868 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58640:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:27:43.104228 20868 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:27:43.127513 21737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:43.117554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:43007 (local address 127.0.0.1:52946)
0808 23:27:43.118329 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:43.118346 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:43.118477 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:43.119584 (+ 1107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:43.119592 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:43.119621 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:43.119971 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:43.119985 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.121968 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.121974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:43.124437 (+ 2463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:43.124454 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.126591 (+ 2137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.126598 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:43.126795 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:43.126843 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 23:27:43.126973 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 23:27:43.127145 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":747,"threads_started":1}
I20260808 23:27:43.128916 21738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:43.118159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52946 (local address 127.9.44.67:43007)
0808 23:27:43.119079 (+ 920us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:43.119085 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:43.119112 (+ 27us) server_negotiation.cc:407] Connection header received
0808 23:27:43.119185 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:43.119190 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:43.119380 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:43.119544 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:43.120209 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.121795 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.124622 (+ 2827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.126070 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.126196 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:43.126430 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:43.127936 (+ 1506us) server_negotiation.cc:299] Negotiation successful
0808 23:27:43.128152 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":789,"thread_start_us":591,"threads_started":1}
I20260808 23:27:43.130995 21735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:43.116995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:45343 (local address 127.0.0.1:52842)
0808 23:27:43.117625 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:43.117672 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:43.117840 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:43.122012 (+ 4172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:43.122016 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:43.122045 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:43.122539 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:43.122554 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.125540 (+ 2986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.125545 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:43.128721 (+ 3176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:43.128733 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.130280 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.130286 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:43.130456 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:43.130489 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 23:27:43.130578 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 23:27:43.130690 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":478,"threads_started":1}
I20260808 23:27:43.131647 21736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:43.117326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52842 (local address 127.9.44.66:45343)
0808 23:27:43.121259 (+ 3933us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:43.121267 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:43.121294 (+ 27us) server_negotiation.cc:407] Connection header received
0808 23:27:43.121395 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:43.121402 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:43.121664 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:43.121847 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:43.122904 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.125372 (+ 2468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.128878 (+ 3506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.130090 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.130207 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:43.130407 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:43.131213 (+ 806us) server_negotiation.cc:299] Negotiation successful
0808 23:27:43.131380 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3780,"thread_start_us":3654,"threads_started":1}
I20260808 23:27:43.132593 21594 tablet_service.cc:1467] Processing CreateTablet for tablet 512ebdcf4d954a0aba00e0bbdcd92613 (DEFAULT_TABLE table=test-table [id=382a5cedda994dd7a2139ca56d519747]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:27:43.134313 21594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 512ebdcf4d954a0aba00e0bbdcd92613. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:43.136466 21737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:43.118225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:43811 (local address 127.0.0.1:40124)
0808 23:27:43.128552 (+ 10327us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:43.128569 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:43.128663 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:43.131403 (+ 2740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:43.131407 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:43.131433 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:43.131581 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:43.131591 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.133014 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.133019 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:43.134569 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:43.134582 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.135794 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.135799 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:43.135990 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:43.136022 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 23:27:43.136103 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 23:27:43.136227 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10239,"thread_start_us":11255,"threads_started":1}
I20260808 23:27:43.136837 21741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:43.118555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40124 (local address 127.9.44.65:43811)
0808 23:27:43.130840 (+ 12285us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:43.130848 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:43.130874 (+ 26us) server_negotiation.cc:407] Connection header received
0808 23:27:43.130967 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:43.130974 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:43.131204 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:43.131405 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:43.131811 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.132887 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.134717 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.135618 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.135767 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:43.135967 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:43.136155 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 23:27:43.136312 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12131,"thread_start_us":11998,"threads_started":1}
I20260808 23:27:43.136430 21354 tablet_service.cc:1467] Processing CreateTablet for tablet 512ebdcf4d954a0aba00e0bbdcd92613 (DEFAULT_TABLE table=test-table [id=382a5cedda994dd7a2139ca56d519747]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:27:43.137698 21354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 512ebdcf4d954a0aba00e0bbdcd92613. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:43.139711 21120 tablet_service.cc:1467] Processing CreateTablet for tablet 512ebdcf4d954a0aba00e0bbdcd92613 (DEFAULT_TABLE table=test-table [id=382a5cedda994dd7a2139ca56d519747]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:27:43.140769 21120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 512ebdcf4d954a0aba00e0bbdcd92613. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:43.148905 21742 tablet_bootstrap.cc:492] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72: Bootstrap starting.
I20260808 23:27:43.151516 21743 tablet_bootstrap.cc:492] T 512ebdcf4d954a0aba00e0bbdcd92613 P d0341bb2dab042899a597a550f919011: Bootstrap starting.
I20260808 23:27:43.152711 21742 tablet_bootstrap.cc:654] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:43.154080 21742 log.cc:824] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:43.154129 21743 tablet_bootstrap.cc:654] T 512ebdcf4d954a0aba00e0bbdcd92613 P d0341bb2dab042899a597a550f919011: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:43.155336 21743 log.cc:824] T 512ebdcf4d954a0aba00e0bbdcd92613 P d0341bb2dab042899a597a550f919011: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:43.157823 21743 tablet_bootstrap.cc:492] T 512ebdcf4d954a0aba00e0bbdcd92613 P d0341bb2dab042899a597a550f919011: No bootstrap required, opened a new log
I20260808 23:27:43.158051 21743 ts_tablet_manager.cc:1397] T 512ebdcf4d954a0aba00e0bbdcd92613 P d0341bb2dab042899a597a550f919011: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.001s
I20260808 23:27:43.159828 21744 tablet_bootstrap.cc:492] T 512ebdcf4d954a0aba00e0bbdcd92613 P adec41244a7d4304aba0dcac828d454d: Bootstrap starting.
I20260808 23:27:43.162544 21742 tablet_bootstrap.cc:492] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72: No bootstrap required, opened a new log
I20260808 23:27:43.162811 21742 ts_tablet_manager.cc:1397] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.008s
I20260808 23:27:43.163105 21744 tablet_bootstrap.cc:654] T 512ebdcf4d954a0aba00e0bbdcd92613 P adec41244a7d4304aba0dcac828d454d: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:43.164355 21744 log.cc:824] T 512ebdcf4d954a0aba00e0bbdcd92613 P adec41244a7d4304aba0dcac828d454d: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:43.165239 21743 raft_consensus.cc:348] T 512ebdcf4d954a0aba00e0bbdcd92613 P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:43.165657 21743 raft_consensus.cc:374] T 512ebdcf4d954a0aba00e0bbdcd92613 P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:43.165772 21743 raft_consensus.cc:729] T 512ebdcf4d954a0aba00e0bbdcd92613 P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0341bb2dab042899a597a550f919011, State: Initialized, Role: FOLLOWER
I20260808 23:27:43.166388 21743 consensus_queue.cc:260] T 512ebdcf4d954a0aba00e0bbdcd92613 P d0341bb2dab042899a597a550f919011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:43.167462 21473 heartbeater.cc:503] Master 127.9.44.126:43315 was elected leader, sending a full tablet report...
I20260808 23:27:43.167937 21743 ts_tablet_manager.cc:1428] T 512ebdcf4d954a0aba00e0bbdcd92613 P d0341bb2dab042899a597a550f919011: Time spent starting tablet: real 0.010s user 0.005s sys 0.006s
I20260808 23:27:43.168257 21744 tablet_bootstrap.cc:492] T 512ebdcf4d954a0aba00e0bbdcd92613 P adec41244a7d4304aba0dcac828d454d: No bootstrap required, opened a new log
I20260808 23:27:43.168507 21744 ts_tablet_manager.cc:1397] T 512ebdcf4d954a0aba00e0bbdcd92613 P adec41244a7d4304aba0dcac828d454d: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260808 23:27:43.172919 21742 raft_consensus.cc:348] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:43.173334 21742 raft_consensus.cc:374] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:43.173429 21742 raft_consensus.cc:729] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3c4f5915fcc462194b8ee854a3bee72, State: Initialized, Role: FOLLOWER
I20260808 23:27:43.173959 21742 consensus_queue.cc:260] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:43.174927 21706 heartbeater.cc:503] Master 127.9.44.126:43315 was elected leader, sending a full tablet report...
I20260808 23:27:43.175424 21742 ts_tablet_manager.cc:1428] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72: Time spent starting tablet: real 0.012s user 0.006s sys 0.007s
I20260808 23:27:43.176033 21744 raft_consensus.cc:348] T 512ebdcf4d954a0aba00e0bbdcd92613 P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:43.176415 21744 raft_consensus.cc:374] T 512ebdcf4d954a0aba00e0bbdcd92613 P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:43.176515 21744 raft_consensus.cc:729] T 512ebdcf4d954a0aba00e0bbdcd92613 P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adec41244a7d4304aba0dcac828d454d, State: Initialized, Role: FOLLOWER
I20260808 23:27:43.176978 21744 consensus_queue.cc:260] T 512ebdcf4d954a0aba00e0bbdcd92613 P adec41244a7d4304aba0dcac828d454d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:43.178476 21744 ts_tablet_manager.cc:1428] T 512ebdcf4d954a0aba00e0bbdcd92613 P adec41244a7d4304aba0dcac828d454d: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
W20260808 23:27:43.226629 21234 tablet.cc:2369] T 512ebdcf4d954a0aba00e0bbdcd92613 P adec41244a7d4304aba0dcac828d454d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:27:43.265733 21707 tablet.cc:2369] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:27:43.271137 21474 tablet.cc:2369] T 512ebdcf4d954a0aba00e0bbdcd92613 P d0341bb2dab042899a597a550f919011: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:27:43.344502 21750 raft_consensus.cc:482] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:27:43.344770 21750 raft_consensus.cc:504] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:43.346524 21750 leader_election.cc:288] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0341bb2dab042899a597a550f919011 (127.9.44.66:45343), adec41244a7d4304aba0dcac828d454d (127.9.44.65:43811)
I20260808 23:27:43.353204 21736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:43.346869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:44969 (local address 127.9.44.66:45343)
0808 23:27:43.347094 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:43.347101 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:43.347124 (+ 23us) server_negotiation.cc:407] Connection header received
0808 23:27:43.347321 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:43.347326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:43.347424 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:43.347524 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:43.348055 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.349125 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.351198 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.352406 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.352514 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:43.352596 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:43.352924 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 23:27:43.353012 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 23:27:43.353302 21520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:43.346864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:45343 (local address 127.9.44.67:44969)
0808 23:27:43.347060 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:43.347077 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:43.347174 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:43.347581 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:43.347586 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:43.347618 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:43.347855 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:43.347871 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.349304 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.349310 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:43.351029 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:43.351041 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.352561 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.352567 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:43.352776 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:43.352813 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 23:27:43.352908 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 23:27:43.353038 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 23:27:43.354216 21385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512ebdcf4d954a0aba00e0bbdcd92613" candidate_uuid: "e3c4f5915fcc462194b8ee854a3bee72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0341bb2dab042899a597a550f919011" is_pre_election: true
I20260808 23:27:43.354409 21741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:43.347424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:35989 (local address 127.9.44.65:43811)
0808 23:27:43.347603 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:43.347609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:43.348409 (+ 800us) server_negotiation.cc:407] Connection header received
0808 23:27:43.348551 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:43.348556 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:43.348649 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:43.348762 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:43.349304 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.350376 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.352434 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.353569 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.353681 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:43.353752 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:43.354061 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 23:27:43.354182 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 23:27:43.354445 21752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:43.347318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:43811 (local address 127.9.44.67:35989)
0808 23:27:43.348297 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:43.348314 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:43.348438 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:43.348774 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:43.348779 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:43.348807 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:43.349148 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:43.349159 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.350500 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.350503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:43.352284 (+ 1781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:43.352300 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.353770 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.353776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:43.353909 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:43.353932 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 23:27:43.354014 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 23:27:43.354128 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":858,"thread_start_us":649,"threads_started":1}
I20260808 23:27:43.354748 21385 raft_consensus.cc:2454] T 512ebdcf4d954a0aba00e0bbdcd92613 P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3c4f5915fcc462194b8ee854a3bee72 in term 0.
I20260808 23:27:43.355381 21511 leader_election.cc:302] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0341bb2dab042899a597a550f919011, e3c4f5915fcc462194b8ee854a3bee72; no voters:
I20260808 23:27:43.355345 21154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512ebdcf4d954a0aba00e0bbdcd92613" candidate_uuid: "e3c4f5915fcc462194b8ee854a3bee72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adec41244a7d4304aba0dcac828d454d" is_pre_election: true
I20260808 23:27:43.355753 21750 raft_consensus.cc:2781] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:27:43.355799 21154 raft_consensus.cc:2454] T 512ebdcf4d954a0aba00e0bbdcd92613 P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3c4f5915fcc462194b8ee854a3bee72 in term 0.
I20260808 23:27:43.355882 21750 raft_consensus.cc:482] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:27:43.355981 21750 raft_consensus.cc:3037] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:43.358183 21750 raft_consensus.cc:504] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:43.358748 21750 leader_election.cc:288] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72 [CANDIDATE]: Term 1 election: Requested vote from peers d0341bb2dab042899a597a550f919011 (127.9.44.66:45343), adec41244a7d4304aba0dcac828d454d (127.9.44.65:43811)
I20260808 23:27:43.359140 21385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512ebdcf4d954a0aba00e0bbdcd92613" candidate_uuid: "e3c4f5915fcc462194b8ee854a3bee72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0341bb2dab042899a597a550f919011"
I20260808 23:27:43.359268 21154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512ebdcf4d954a0aba00e0bbdcd92613" candidate_uuid: "e3c4f5915fcc462194b8ee854a3bee72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adec41244a7d4304aba0dcac828d454d"
I20260808 23:27:43.359311 21385 raft_consensus.cc:3037] T 512ebdcf4d954a0aba00e0bbdcd92613 P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:43.359442 21154 raft_consensus.cc:3037] T 512ebdcf4d954a0aba00e0bbdcd92613 P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:43.361449 21385 raft_consensus.cc:2454] T 512ebdcf4d954a0aba00e0bbdcd92613 P d0341bb2dab042899a597a550f919011 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3c4f5915fcc462194b8ee854a3bee72 in term 1.
I20260808 23:27:43.361545 21154 raft_consensus.cc:2454] T 512ebdcf4d954a0aba00e0bbdcd92613 P adec41244a7d4304aba0dcac828d454d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3c4f5915fcc462194b8ee854a3bee72 in term 1.
I20260808 23:27:43.361804 21511 leader_election.cc:302] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0341bb2dab042899a597a550f919011, e3c4f5915fcc462194b8ee854a3bee72; no voters:
I20260808 23:27:43.362080 21750 raft_consensus.cc:2781] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:27:43.362540 21750 raft_consensus.cc:686] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72 [term 1 LEADER]: Becoming Leader. State: Replica: e3c4f5915fcc462194b8ee854a3bee72, State: Running, Role: LEADER
I20260808 23:27:43.363040 21750 consensus_queue.cc:237] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:43.367274 20868 catalog_manager.cc:5495] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72 reported cstate change: term changed from 0 to 1, leader changed from <none> to e3c4f5915fcc462194b8ee854a3bee72 (127.9.44.67). New cstate: current_term: 1 leader_uuid: "e3c4f5915fcc462194b8ee854a3bee72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } health_report { overall_health: HEALTHY } } }
I20260808 23:27:43.416759 21738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:43.411366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52950 (local address 127.9.44.67:43007)
0808 23:27:43.411603 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:43.411622 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:43.411640 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:27:43.412000 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:43.412005 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:43.412103 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:43.412280 (+ 177us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:27:43.412669 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.413367 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.414672 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.415082 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.415198 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:43.415647 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:27:43.416190 (+ 543us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:27:43.416318 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:43.416431 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 23:27:43.416515 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 23:27:43.424456 21154 raft_consensus.cc:1261] T 512ebdcf4d954a0aba00e0bbdcd92613 P adec41244a7d4304aba0dcac828d454d [term 1 FOLLOWER]: Refusing update from remote peer e3c4f5915fcc462194b8ee854a3bee72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:27:43.424492 21385 raft_consensus.cc:1261] T 512ebdcf4d954a0aba00e0bbdcd92613 P d0341bb2dab042899a597a550f919011 [term 1 FOLLOWER]: Refusing update from remote peer e3c4f5915fcc462194b8ee854a3bee72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:27:43.425307 21758 consensus_queue.cc:1035] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:27:43.425639 21750 consensus_queue.cc:1035] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:27:43.435324 21762 mvcc.cc:205] Tried to move back new op lower bound from 7316404893377449984 to 7316404893138796544. Current Snapshot: MvccSnapshot[applied={T|T < 7316404893377449984}]
I20260808 23:27:43.449252 20870 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:27:43.449456 20870 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:27:43.452625 20870 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167: Sending DeleteTablet for 3 replicas of tablet 512ebdcf4d954a0aba00e0bbdcd92613
I20260808 23:27:43.453634 21354 tablet_service.cc:1514] Processing DeleteTablet for tablet 512ebdcf4d954a0aba00e0bbdcd92613 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:27:43 UTC) from {username='oryx', principal='oryx/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:52842
I20260808 23:27:43.453895 21594 tablet_service.cc:1514] Processing DeleteTablet for tablet 512ebdcf4d954a0aba00e0bbdcd92613 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:27:43 UTC) from {username='oryx', principal='oryx/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:52946
I20260808 23:27:43.454136 21120 tablet_service.cc:1514] Processing DeleteTablet for tablet 512ebdcf4d954a0aba00e0bbdcd92613 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:27:43 UTC) from {username='oryx', principal='oryx/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:40124
I20260808 23:27:43.456032 20870 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58640:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:27:43.457098 20870 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:27:43.466514 21592 tablet_service.cc:1467] Processing CreateTablet for tablet 6ac206fbb47a4e709f04f194d20c9f82 (DEFAULT_TABLE table=test-table [id=7bf582f44ab64f7787c00dc285cfa3b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:27:43.467218 21592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ac206fbb47a4e709f04f194d20c9f82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:43.468859 21776 tablet_replica.cc:330] T 512ebdcf4d954a0aba00e0bbdcd92613 P d0341bb2dab042899a597a550f919011: stopping tablet replica
I20260808 23:27:43.469273 21776 raft_consensus.cc:2229] T 512ebdcf4d954a0aba00e0bbdcd92613 P d0341bb2dab042899a597a550f919011 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:27:43.469731 21776 raft_consensus.cc:2258] T 512ebdcf4d954a0aba00e0bbdcd92613 P d0341bb2dab042899a597a550f919011 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:27:43.471868 21776 ts_tablet_manager.cc:1905] T 512ebdcf4d954a0aba00e0bbdcd92613 P d0341bb2dab042899a597a550f919011: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:27:43.472215 21351 tablet_service.cc:1467] Processing CreateTablet for tablet 6ac206fbb47a4e709f04f194d20c9f82 (DEFAULT_TABLE table=test-table [id=7bf582f44ab64f7787c00dc285cfa3b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:27:43.472693 21351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ac206fbb47a4e709f04f194d20c9f82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:43.476811 21119 tablet_service.cc:1467] Processing CreateTablet for tablet 6ac206fbb47a4e709f04f194d20c9f82 (DEFAULT_TABLE table=test-table [id=7bf582f44ab64f7787c00dc285cfa3b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:27:43.477377 21119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ac206fbb47a4e709f04f194d20c9f82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:43.479041 21776 ts_tablet_manager.cc:1918] T 512ebdcf4d954a0aba00e0bbdcd92613 P d0341bb2dab042899a597a550f919011: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:27:43.479197 21776 log.cc:1196] T 512ebdcf4d954a0aba00e0bbdcd92613 P d0341bb2dab042899a597a550f919011: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/512ebdcf4d954a0aba00e0bbdcd92613
I20260808 23:27:43.479756 21776 ts_tablet_manager.cc:1939] T 512ebdcf4d954a0aba00e0bbdcd92613 P d0341bb2dab042899a597a550f919011: Deleting consensus metadata
I20260808 23:27:43.481499 20846 catalog_manager.cc:4841] TS d0341bb2dab042899a597a550f919011 (127.9.44.66:45343): tablet 512ebdcf4d954a0aba00e0bbdcd92613 (table test-table [id=382a5cedda994dd7a2139ca56d519747]) successfully deleted
I20260808 23:27:43.487610 21743 tablet_bootstrap.cc:492] T 6ac206fbb47a4e709f04f194d20c9f82 P d0341bb2dab042899a597a550f919011: Bootstrap starting.
I20260808 23:27:43.489333 21743 tablet_bootstrap.cc:654] T 6ac206fbb47a4e709f04f194d20c9f82 P d0341bb2dab042899a597a550f919011: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:43.493239 21742 tablet_bootstrap.cc:492] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72: Bootstrap starting.
I20260808 23:27:43.494109 21744 tablet_bootstrap.cc:492] T 6ac206fbb47a4e709f04f194d20c9f82 P adec41244a7d4304aba0dcac828d454d: Bootstrap starting.
I20260808 23:27:43.494933 21742 tablet_bootstrap.cc:654] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:43.496212 21744 tablet_bootstrap.cc:654] T 6ac206fbb47a4e709f04f194d20c9f82 P adec41244a7d4304aba0dcac828d454d: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:43.501806 21779 tablet_replica.cc:330] T 512ebdcf4d954a0aba00e0bbdcd92613 P adec41244a7d4304aba0dcac828d454d: stopping tablet replica
I20260808 23:27:43.502290 21779 raft_consensus.cc:2229] T 512ebdcf4d954a0aba00e0bbdcd92613 P adec41244a7d4304aba0dcac828d454d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:27:43.502802 21779 raft_consensus.cc:2258] T 512ebdcf4d954a0aba00e0bbdcd92613 P adec41244a7d4304aba0dcac828d454d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:27:43.509778 21744 tablet_bootstrap.cc:492] T 6ac206fbb47a4e709f04f194d20c9f82 P adec41244a7d4304aba0dcac828d454d: No bootstrap required, opened a new log
I20260808 23:27:43.509989 21744 ts_tablet_manager.cc:1397] T 6ac206fbb47a4e709f04f194d20c9f82 P adec41244a7d4304aba0dcac828d454d: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.008s
I20260808 23:27:43.510891 21744 raft_consensus.cc:348] T 6ac206fbb47a4e709f04f194d20c9f82 P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:43.511216 21744 raft_consensus.cc:374] T 6ac206fbb47a4e709f04f194d20c9f82 P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:43.511312 21744 raft_consensus.cc:729] T 6ac206fbb47a4e709f04f194d20c9f82 P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adec41244a7d4304aba0dcac828d454d, State: Initialized, Role: FOLLOWER
I20260808 23:27:43.511495 21778 tablet_replica.cc:330] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72: stopping tablet replica
I20260808 23:27:43.511559 21744 consensus_queue.cc:260] T 6ac206fbb47a4e709f04f194d20c9f82 P adec41244a7d4304aba0dcac828d454d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:43.511974 21778 raft_consensus.cc:2229] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:27:43.512717 21778 raft_consensus.cc:2258] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:27:43.513321 21779 ts_tablet_manager.cc:1905] T 512ebdcf4d954a0aba00e0bbdcd92613 P adec41244a7d4304aba0dcac828d454d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:27:43.514746 21778 ts_tablet_manager.cc:1905] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:27:43.515065 21744 ts_tablet_manager.cc:1428] T 6ac206fbb47a4e709f04f194d20c9f82 P adec41244a7d4304aba0dcac828d454d: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260808 23:27:43.516412 21742 tablet_bootstrap.cc:492] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72: No bootstrap required, opened a new log
I20260808 23:27:43.516537 21743 tablet_bootstrap.cc:492] T 6ac206fbb47a4e709f04f194d20c9f82 P d0341bb2dab042899a597a550f919011: No bootstrap required, opened a new log
I20260808 23:27:43.516567 21742 ts_tablet_manager.cc:1397] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72: Time spent bootstrapping tablet: real 0.023s user 0.011s sys 0.007s
I20260808 23:27:43.516772 21743 ts_tablet_manager.cc:1397] T 6ac206fbb47a4e709f04f194d20c9f82 P d0341bb2dab042899a597a550f919011: Time spent bootstrapping tablet: real 0.029s user 0.009s sys 0.003s
I20260808 23:27:43.517344 21742 raft_consensus.cc:348] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:43.517588 21742 raft_consensus.cc:374] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:43.517678 21742 raft_consensus.cc:729] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3c4f5915fcc462194b8ee854a3bee72, State: Initialized, Role: FOLLOWER
I20260808 23:27:43.517683 21743 raft_consensus.cc:348] T 6ac206fbb47a4e709f04f194d20c9f82 P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:43.518018 21743 raft_consensus.cc:374] T 6ac206fbb47a4e709f04f194d20c9f82 P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:43.517946 21742 consensus_queue.cc:260] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:43.518186 21743 raft_consensus.cc:729] T 6ac206fbb47a4e709f04f194d20c9f82 P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0341bb2dab042899a597a550f919011, State: Initialized, Role: FOLLOWER
I20260808 23:27:43.518728 21743 consensus_queue.cc:260] T 6ac206fbb47a4e709f04f194d20c9f82 P d0341bb2dab042899a597a550f919011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:43.519462 21743 ts_tablet_manager.cc:1428] T 6ac206fbb47a4e709f04f194d20c9f82 P d0341bb2dab042899a597a550f919011: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260808 23:27:43.520574 21742 ts_tablet_manager.cc:1428] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260808 23:27:43.525125 21778 ts_tablet_manager.cc:1918] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:27:43.525120 21779 ts_tablet_manager.cc:1918] T 512ebdcf4d954a0aba00e0bbdcd92613 P adec41244a7d4304aba0dcac828d454d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:27:43.525302 21778 log.cc:1196] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/512ebdcf4d954a0aba00e0bbdcd92613
I20260808 23:27:43.525424 21779 log.cc:1196] T 512ebdcf4d954a0aba00e0bbdcd92613 P adec41244a7d4304aba0dcac828d454d: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-0/wal/wals/512ebdcf4d954a0aba00e0bbdcd92613
I20260808 23:27:43.525826 21778 ts_tablet_manager.cc:1939] T 512ebdcf4d954a0aba00e0bbdcd92613 P e3c4f5915fcc462194b8ee854a3bee72: Deleting consensus metadata
I20260808 23:27:43.526113 21779 ts_tablet_manager.cc:1939] T 512ebdcf4d954a0aba00e0bbdcd92613 P adec41244a7d4304aba0dcac828d454d: Deleting consensus metadata
I20260808 23:27:43.527482 20847 catalog_manager.cc:4841] TS e3c4f5915fcc462194b8ee854a3bee72 (127.9.44.67:43007): tablet 512ebdcf4d954a0aba00e0bbdcd92613 (table test-table [id=382a5cedda994dd7a2139ca56d519747]) successfully deleted
I20260808 23:27:43.527616 20846 catalog_manager.cc:4841] TS adec41244a7d4304aba0dcac828d454d (127.9.44.65:43811): tablet 512ebdcf4d954a0aba00e0bbdcd92613 (table test-table [id=382a5cedda994dd7a2139ca56d519747]) successfully deleted
I20260808 23:27:43.587327 21750 raft_consensus.cc:482] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:27:43.587524 21750 raft_consensus.cc:504] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:43.588194 21750 leader_election.cc:288] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adec41244a7d4304aba0dcac828d454d (127.9.44.65:43811), d0341bb2dab042899a597a550f919011 (127.9.44.66:45343)
I20260808 23:27:43.588729 21154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ac206fbb47a4e709f04f194d20c9f82" candidate_uuid: "e3c4f5915fcc462194b8ee854a3bee72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adec41244a7d4304aba0dcac828d454d" is_pre_election: true
I20260808 23:27:43.588838 21385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ac206fbb47a4e709f04f194d20c9f82" candidate_uuid: "e3c4f5915fcc462194b8ee854a3bee72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0341bb2dab042899a597a550f919011" is_pre_election: true
I20260808 23:27:43.588980 21154 raft_consensus.cc:2454] T 6ac206fbb47a4e709f04f194d20c9f82 P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3c4f5915fcc462194b8ee854a3bee72 in term 0.
I20260808 23:27:43.589109 21385 raft_consensus.cc:2454] T 6ac206fbb47a4e709f04f194d20c9f82 P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3c4f5915fcc462194b8ee854a3bee72 in term 0.
I20260808 23:27:43.589421 21511 leader_election.cc:302] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adec41244a7d4304aba0dcac828d454d, e3c4f5915fcc462194b8ee854a3bee72; no voters:
I20260808 23:27:43.589691 21750 raft_consensus.cc:2781] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:27:43.589820 21750 raft_consensus.cc:482] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:27:43.589929 21750 raft_consensus.cc:3037] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:43.592123 21750 raft_consensus.cc:504] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:43.592672 21750 leader_election.cc:288] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72 [CANDIDATE]: Term 1 election: Requested vote from peers adec41244a7d4304aba0dcac828d454d (127.9.44.65:43811), d0341bb2dab042899a597a550f919011 (127.9.44.66:45343)
I20260808 23:27:43.593101 21154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ac206fbb47a4e709f04f194d20c9f82" candidate_uuid: "e3c4f5915fcc462194b8ee854a3bee72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adec41244a7d4304aba0dcac828d454d"
I20260808 23:27:43.593187 21385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ac206fbb47a4e709f04f194d20c9f82" candidate_uuid: "e3c4f5915fcc462194b8ee854a3bee72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0341bb2dab042899a597a550f919011"
I20260808 23:27:43.593307 21154 raft_consensus.cc:3037] T 6ac206fbb47a4e709f04f194d20c9f82 P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:43.593389 21385 raft_consensus.cc:3037] T 6ac206fbb47a4e709f04f194d20c9f82 P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:43.595640 21385 raft_consensus.cc:2454] T 6ac206fbb47a4e709f04f194d20c9f82 P d0341bb2dab042899a597a550f919011 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3c4f5915fcc462194b8ee854a3bee72 in term 1.
I20260808 23:27:43.596042 21511 leader_election.cc:302] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0341bb2dab042899a597a550f919011, e3c4f5915fcc462194b8ee854a3bee72; no voters:
I20260808 23:27:43.596248 21154 raft_consensus.cc:2454] T 6ac206fbb47a4e709f04f194d20c9f82 P adec41244a7d4304aba0dcac828d454d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3c4f5915fcc462194b8ee854a3bee72 in term 1.
I20260808 23:27:43.596323 21750 raft_consensus.cc:2781] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:27:43.596469 21750 raft_consensus.cc:686] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72 [term 1 LEADER]: Becoming Leader. State: Replica: e3c4f5915fcc462194b8ee854a3bee72, State: Running, Role: LEADER
I20260808 23:27:43.596843 21750 consensus_queue.cc:237] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:43.599432 20868 catalog_manager.cc:5495] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72 reported cstate change: term changed from 0 to 1, leader changed from <none> to e3c4f5915fcc462194b8ee854a3bee72 (127.9.44.67). New cstate: current_term: 1 leader_uuid: "e3c4f5915fcc462194b8ee854a3bee72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } health_report { overall_health: HEALTHY } } }
Aug 08 23:27:43 dist-test-slave-6bbx krb5kdc[20787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231661, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.126@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM
I20260808 23:27:43.642915 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:43.630122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:43315 (local address 127.0.0.1:58654)
0808 23:27:43.630717 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:43.630735 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:43.630869 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:43.631163 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:43.631168 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:43.631890 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:43.632082 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:43.632092 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.633122 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.633127 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:43.634114 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:43.634127 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.634871 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.634878 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:43.635023 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:43.635694 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:43.635719 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:43.637797 (+ 2078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:43.640549 (+ 2752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:43.640561 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:43.640570 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:43.640968 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:43.641304 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:43.641309 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:43.641313 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:43.641467 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:43.641940 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:43.641948 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:43.642129 (+ 181us) client_negotiation.cc:769] Sending connection context
0808 23:27:43.642270 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 23:27:43.642532 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":364,"threads_started":1}
I20260808 23:27:43.643029 21800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:43.629981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58654 (local address 127.9.44.126:43315)
0808 23:27:43.630455 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:43.630462 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:43.630822 (+ 360us) server_negotiation.cc:407] Connection header received
0808 23:27:43.630978 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:43.630983 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:43.631040 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:43.631124 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:43.632240 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.632994 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.634247 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:43.634699 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:43.634807 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:43.637969 (+ 3162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:43.637974 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:43.637977 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:43.638004 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:43.640433 (+ 2429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:43.641075 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:43.641081 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:43.641083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:43.641205 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:43.641566 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:43.641571 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:43.641574 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:43.641814 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:43.641968 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:43.642380 (+ 412us) server_negotiation.cc:299] Negotiation successful
0808 23:27:43.642679 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":100,"server-negotiator.queue_time_us":330,"thread_start_us":223,"threads_started":1}
I20260808 23:27:43.648341 20868 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:58654:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 23:27:43.649371 20868 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:27:43.655812 21120 tablet_service.cc:1467] Processing CreateTablet for tablet f5a0f972913f42fdaeb07c60e1bc4acd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1f18a0bd8362407fb0cd0dcef953ba76]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 23:27:43.655983 21351 tablet_service.cc:1467] Processing CreateTablet for tablet f5a0f972913f42fdaeb07c60e1bc4acd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1f18a0bd8362407fb0cd0dcef953ba76]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 23:27:43.656396 21120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5a0f972913f42fdaeb07c60e1bc4acd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:43.656481 21351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5a0f972913f42fdaeb07c60e1bc4acd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:43.660522 21594 tablet_service.cc:1467] Processing CreateTablet for tablet f5a0f972913f42fdaeb07c60e1bc4acd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1f18a0bd8362407fb0cd0dcef953ba76]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 23:27:43.661096 21594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5a0f972913f42fdaeb07c60e1bc4acd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:43.662777 21743 tablet_bootstrap.cc:492] T f5a0f972913f42fdaeb07c60e1bc4acd P d0341bb2dab042899a597a550f919011: Bootstrap starting.
I20260808 23:27:43.664467 21743 tablet_bootstrap.cc:654] T f5a0f972913f42fdaeb07c60e1bc4acd P d0341bb2dab042899a597a550f919011: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:43.665001 21744 tablet_bootstrap.cc:492] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d: Bootstrap starting.
I20260808 23:27:43.666692 21744 tablet_bootstrap.cc:654] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:43.668996 21742 tablet_bootstrap.cc:492] T f5a0f972913f42fdaeb07c60e1bc4acd P e3c4f5915fcc462194b8ee854a3bee72: Bootstrap starting.
I20260808 23:27:43.670617 21742 tablet_bootstrap.cc:654] T f5a0f972913f42fdaeb07c60e1bc4acd P e3c4f5915fcc462194b8ee854a3bee72: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:43.670857 21744 tablet_bootstrap.cc:492] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d: No bootstrap required, opened a new log
I20260808 23:27:43.670882 21743 tablet_bootstrap.cc:492] T f5a0f972913f42fdaeb07c60e1bc4acd P d0341bb2dab042899a597a550f919011: No bootstrap required, opened a new log
I20260808 23:27:43.671012 21744 ts_tablet_manager.cc:1397] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260808 23:27:43.671036 21743 ts_tablet_manager.cc:1397] T f5a0f972913f42fdaeb07c60e1bc4acd P d0341bb2dab042899a597a550f919011: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260808 23:27:43.671696 21744 raft_consensus.cc:348] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:43.671937 21744 raft_consensus.cc:374] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:43.672013 21744 raft_consensus.cc:729] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adec41244a7d4304aba0dcac828d454d, State: Initialized, Role: FOLLOWER
I20260808 23:27:43.672278 21744 consensus_queue.cc:260] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:43.672683 21751 tablet_replica.cc:439] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } } }
I20260808 23:27:43.672840 21743 raft_consensus.cc:348] T f5a0f972913f42fdaeb07c60e1bc4acd P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:43.673030 21751 tablet_replica.cc:442] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:27:43.673089 21743 raft_consensus.cc:374] T f5a0f972913f42fdaeb07c60e1bc4acd P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:43.673167 21743 raft_consensus.cc:729] T f5a0f972913f42fdaeb07c60e1bc4acd P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0341bb2dab042899a597a550f919011, State: Initialized, Role: FOLLOWER
I20260808 23:27:43.673465 21743 consensus_queue.cc:260] T f5a0f972913f42fdaeb07c60e1bc4acd P d0341bb2dab042899a597a550f919011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:43.673900 21748 tablet_replica.cc:439] T f5a0f972913f42fdaeb07c60e1bc4acd P d0341bb2dab042899a597a550f919011: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } } }
I20260808 23:27:43.674297 21748 tablet_replica.cc:442] T f5a0f972913f42fdaeb07c60e1bc4acd P d0341bb2dab042899a597a550f919011: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:27:43.674744 21743 ts_tablet_manager.cc:1428] T f5a0f972913f42fdaeb07c60e1bc4acd P d0341bb2dab042899a597a550f919011: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260808 23:27:43.677809 21744 ts_tablet_manager.cc:1428] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260808 23:27:43.685104 21742 tablet_bootstrap.cc:492] T f5a0f972913f42fdaeb07c60e1bc4acd P e3c4f5915fcc462194b8ee854a3bee72: No bootstrap required, opened a new log
I20260808 23:27:43.685267 21742 ts_tablet_manager.cc:1397] T f5a0f972913f42fdaeb07c60e1bc4acd P e3c4f5915fcc462194b8ee854a3bee72: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
I20260808 23:27:43.685981 21742 raft_consensus.cc:348] T f5a0f972913f42fdaeb07c60e1bc4acd P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:43.686180 21742 raft_consensus.cc:374] T f5a0f972913f42fdaeb07c60e1bc4acd P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:43.686283 21742 raft_consensus.cc:729] T f5a0f972913f42fdaeb07c60e1bc4acd P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3c4f5915fcc462194b8ee854a3bee72, State: Initialized, Role: FOLLOWER
I20260808 23:27:43.686537 21742 consensus_queue.cc:260] T f5a0f972913f42fdaeb07c60e1bc4acd P e3c4f5915fcc462194b8ee854a3bee72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:43.687089 21742 ts_tablet_manager.cc:1428] T f5a0f972913f42fdaeb07c60e1bc4acd P e3c4f5915fcc462194b8ee854a3bee72: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260808 23:27:43.686983 21750 tablet_replica.cc:439] T f5a0f972913f42fdaeb07c60e1bc4acd P e3c4f5915fcc462194b8ee854a3bee72: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } } }
I20260808 23:27:43.687332 21750 tablet_replica.cc:442] T f5a0f972913f42fdaeb07c60e1bc4acd P e3c4f5915fcc462194b8ee854a3bee72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:27:44.066411 21751 raft_consensus.cc:482] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:27:44.066601 21751 raft_consensus.cc:504] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:44.067984 21751 leader_election.cc:288] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0341bb2dab042899a597a550f919011 (127.9.44.66:45343), e3c4f5915fcc462194b8ee854a3bee72 (127.9.44.67:43007)
I20260808 23:27:44.081610 21750 consensus_queue.cc:1035] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:27:44.087474 21813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:44.068282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:45343 (local address 127.9.44.65:52761)
0808 23:27:44.080253 (+ 11971us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:44.080276 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:44.080440 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:44.081537 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:44.081542 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:44.081570 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:44.081845 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:44.081858 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.083180 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.083186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:44.085012 (+ 1826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:44.085031 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.086661 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.086668 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:44.086873 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:44.086910 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 23:27:44.087000 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 23:27:44.087146 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11803,"thread_start_us":13085,"threads_started":1}
I20260808 23:27:44.088270 21814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:44.068318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:52761 (local address 127.9.44.66:45343)
0808 23:27:44.080900 (+ 12582us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:44.080907 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:44.080927 (+ 20us) server_negotiation.cc:407] Connection header received
0808 23:27:44.080996 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:44.081000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:44.081076 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:44.081176 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:44.081988 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.083001 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.085210 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.086324 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.086451 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:44.086553 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:44.087799 (+ 1246us) server_negotiation.cc:299] Negotiation successful
0808 23:27:44.087952 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12450,"thread_start_us":12317,"threads_started":1}
I20260808 23:27:44.088297 21812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:44.068312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:43007 (local address 127.9.44.65:47375)
0808 23:27:44.081079 (+ 12767us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:44.081097 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:44.081215 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:44.082185 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:44.082190 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:44.082216 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:44.082564 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:44.082576 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.084060 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.084066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:44.086035 (+ 1969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:44.086052 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.087579 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.087585 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:44.087769 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:44.087800 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 23:27:44.087865 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 23:27:44.087985 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12650,"thread_start_us":12988,"threads_started":1}
I20260808 23:27:44.088892 21771 consensus_queue.cc:1035] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:27:44.088905 21383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5a0f972913f42fdaeb07c60e1bc4acd" candidate_uuid: "adec41244a7d4304aba0dcac828d454d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0341bb2dab042899a597a550f919011" is_pre_election: true
I20260808 23:27:44.089242 21383 raft_consensus.cc:2454] T f5a0f972913f42fdaeb07c60e1bc4acd P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adec41244a7d4304aba0dcac828d454d in term 0.
I20260808 23:27:44.089428 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:44.068384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:47375 (local address 127.9.44.67:43007)
0808 23:27:44.081856 (+ 13472us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:44.081861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:44.081879 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:27:44.081939 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:44.081943 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:44.082009 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:44.082706 (+ 697us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:44.082883 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.083894 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.086208 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.087344 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.088826 (+ 1482us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:44.088939 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:44.088987 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 23:27:44.089120 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13366,"thread_start_us":13260,"threads_started":1}
I20260808 23:27:44.090126 21030 leader_election.cc:302] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adec41244a7d4304aba0dcac828d454d, d0341bb2dab042899a597a550f919011; no voters:
I20260808 23:27:44.090173 21631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5a0f972913f42fdaeb07c60e1bc4acd" candidate_uuid: "adec41244a7d4304aba0dcac828d454d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3c4f5915fcc462194b8ee854a3bee72" is_pre_election: true
I20260808 23:27:44.090615 21751 raft_consensus.cc:2781] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:27:44.090576 21631 raft_consensus.cc:2454] T f5a0f972913f42fdaeb07c60e1bc4acd P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adec41244a7d4304aba0dcac828d454d in term 0.
I20260808 23:27:44.090727 21751 raft_consensus.cc:482] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:27:44.090827 21751 raft_consensus.cc:3037] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:44.093696 21751 raft_consensus.cc:504] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:44.094295 21751 leader_election.cc:288] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d [CANDIDATE]: Term 1 election: Requested vote from peers d0341bb2dab042899a597a550f919011 (127.9.44.66:45343), e3c4f5915fcc462194b8ee854a3bee72 (127.9.44.67:43007)
I20260808 23:27:44.094669 21383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5a0f972913f42fdaeb07c60e1bc4acd" candidate_uuid: "adec41244a7d4304aba0dcac828d454d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0341bb2dab042899a597a550f919011"
I20260808 23:27:44.094717 21631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5a0f972913f42fdaeb07c60e1bc4acd" candidate_uuid: "adec41244a7d4304aba0dcac828d454d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3c4f5915fcc462194b8ee854a3bee72"
I20260808 23:27:44.094873 21383 raft_consensus.cc:3037] T f5a0f972913f42fdaeb07c60e1bc4acd P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:44.094877 21631 raft_consensus.cc:3037] T f5a0f972913f42fdaeb07c60e1bc4acd P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:44.097131 21383 raft_consensus.cc:2454] T f5a0f972913f42fdaeb07c60e1bc4acd P d0341bb2dab042899a597a550f919011 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adec41244a7d4304aba0dcac828d454d in term 1.
I20260808 23:27:44.097293 21631 raft_consensus.cc:2454] T f5a0f972913f42fdaeb07c60e1bc4acd P e3c4f5915fcc462194b8ee854a3bee72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adec41244a7d4304aba0dcac828d454d in term 1.
I20260808 23:27:44.097560 21030 leader_election.cc:302] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adec41244a7d4304aba0dcac828d454d, d0341bb2dab042899a597a550f919011; no voters:
I20260808 23:27:44.097821 21751 raft_consensus.cc:2781] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:27:44.097980 21751 raft_consensus.cc:686] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d [term 1 LEADER]: Becoming Leader. State: Replica: adec41244a7d4304aba0dcac828d454d, State: Running, Role: LEADER
I20260808 23:27:44.098304 21751 consensus_queue.cc:237] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:44.100157 21830 tablet_replica.cc:439] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d: TxnStatusTablet state changed. Reason: New leader adec41244a7d4304aba0dcac828d454d. Latest consensus state: current_term: 1 leader_uuid: "adec41244a7d4304aba0dcac828d454d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } } }
I20260808 23:27:44.100502 21830 tablet_replica.cc:442] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d: This TxnStatusTablet replica's current role is: LEADER
I20260808 23:27:44.101150 21836 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 23:27:44.101375 21836 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 23:27:44.101642 20868 catalog_manager.cc:5495] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d reported cstate change: term changed from 0 to 1, leader changed from <none> to adec41244a7d4304aba0dcac828d454d (127.9.44.65). New cstate: current_term: 1 leader_uuid: "adec41244a7d4304aba0dcac828d454d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } health_report { overall_health: UNKNOWN } } }
I20260808 23:27:44.130201 21839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:44.123865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40128 (local address 127.9.44.65:43811)
0808 23:27:44.124451 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:44.124456 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:44.124477 (+ 21us) server_negotiation.cc:407] Connection header received
0808 23:27:44.124544 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:44.124547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:44.124617 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:44.124733 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:27:44.125154 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.125984 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.127920 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.128406 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.128505 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:44.129098 (+ 593us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:27:44.129625 (+ 527us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:27:44.129727 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:44.129869 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 23:27:44.129929 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":335,"threads_started":1}
I20260808 23:27:44.130290 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:44.123869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:43811 (local address 127.0.0.1:40128)
0808 23:27:44.124111 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:44.124132 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:44.124268 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:44.124792 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:44.124797 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:44.124810 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 23:27:44.125003 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:44.125015 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.126104 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.126110 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:44.127786 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:44.127803 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.128550 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.128557 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:44.128785 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:44.128866 (+ 81us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 23:27:44.129734 (+ 868us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 23:27:44.129757 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 23:27:44.129847 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 23:27:44.129997 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 23:27:44.134481 21383 raft_consensus.cc:1261] T f5a0f972913f42fdaeb07c60e1bc4acd P d0341bb2dab042899a597a550f919011 [term 1 FOLLOWER]: Refusing update from remote peer adec41244a7d4304aba0dcac828d454d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:27:44.134563 21631 raft_consensus.cc:1261] T f5a0f972913f42fdaeb07c60e1bc4acd P e3c4f5915fcc462194b8ee854a3bee72 [term 1 FOLLOWER]: Refusing update from remote peer adec41244a7d4304aba0dcac828d454d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:27:44.135239 21751 consensus_queue.cc:1035] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:27:44.135535 21830 consensus_queue.cc:1035] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:27:44.138298 21771 tablet_replica.cc:439] T f5a0f972913f42fdaeb07c60e1bc4acd P e3c4f5915fcc462194b8ee854a3bee72: TxnStatusTablet state changed. Reason: New leader adec41244a7d4304aba0dcac828d454d. Latest consensus state: current_term: 1 leader_uuid: "adec41244a7d4304aba0dcac828d454d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } } }
I20260808 23:27:44.138633 21771 tablet_replica.cc:442] T f5a0f972913f42fdaeb07c60e1bc4acd P e3c4f5915fcc462194b8ee854a3bee72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:27:44.142206 21830 tablet_replica.cc:439] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "adec41244a7d4304aba0dcac828d454d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } } }
I20260808 23:27:44.142462 21830 tablet_replica.cc:442] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d: This TxnStatusTablet replica's current role is: LEADER
I20260808 23:27:44.149639 21748 tablet_replica.cc:439] T f5a0f972913f42fdaeb07c60e1bc4acd P d0341bb2dab042899a597a550f919011: TxnStatusTablet state changed. Reason: New leader adec41244a7d4304aba0dcac828d454d. Latest consensus state: current_term: 1 leader_uuid: "adec41244a7d4304aba0dcac828d454d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } } }
I20260808 23:27:44.149920 21748 tablet_replica.cc:442] T f5a0f972913f42fdaeb07c60e1bc4acd P d0341bb2dab042899a597a550f919011: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:27:44.150475 21771 tablet_replica.cc:439] T f5a0f972913f42fdaeb07c60e1bc4acd P e3c4f5915fcc462194b8ee854a3bee72: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "adec41244a7d4304aba0dcac828d454d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } } }
I20260808 23:27:44.150708 21771 tablet_replica.cc:442] T f5a0f972913f42fdaeb07c60e1bc4acd P e3c4f5915fcc462194b8ee854a3bee72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:27:44.150640 21751 tablet_replica.cc:439] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "adec41244a7d4304aba0dcac828d454d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } } }
I20260808 23:27:44.150889 21751 tablet_replica.cc:442] T f5a0f972913f42fdaeb07c60e1bc4acd P adec41244a7d4304aba0dcac828d454d: This TxnStatusTablet replica's current role is: LEADER
I20260808 23:27:44.155800 21748 tablet_replica.cc:439] T f5a0f972913f42fdaeb07c60e1bc4acd P d0341bb2dab042899a597a550f919011: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "adec41244a7d4304aba0dcac828d454d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } } }
I20260808 23:27:44.156030 21748 tablet_replica.cc:442] T f5a0f972913f42fdaeb07c60e1bc4acd P d0341bb2dab042899a597a550f919011: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 23:27:44.170274 21839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:44.165027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:43729 (local address 127.9.44.65:43811)
0808 23:27:44.165164 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:44.165170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:44.165663 (+ 493us) server_negotiation.cc:407] Connection header received
0808 23:27:44.165838 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:44.165841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:44.165907 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:44.165983 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:27:44.166460 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.167199 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.168595 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.169068 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.169178 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:44.169590 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:27:44.169684 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:27:44.169756 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:44.169907 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 23:27:44.169984 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 23:27:44.170370 21855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:44.164908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:43811 (local address 127.9.44.67:43729)
0808 23:27:44.165563 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:44.165578 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:44.165718 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:44.166011 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:44.166016 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:44.166029 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 23:27:44.166297 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:44.166308 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.167340 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.167344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:44.168448 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:44.168462 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.169227 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.169233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:44.169372 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:44.169426 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 23:27:44.169776 (+ 350us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 23:27:44.169788 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 23:27:44.169855 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 23:27:44.169987 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":386,"threads_started":1}
I20260808 23:27:44.214851 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:44.209132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:48093 (local address 127.9.44.67:43007)
0808 23:27:44.209338 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:44.209344 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:44.209493 (+ 149us) server_negotiation.cc:407] Connection header received
0808 23:27:44.209741 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:44.209747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:44.209813 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:44.209896 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:27:44.210596 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.211383 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.213163 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.213654 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.213728 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:44.214206 (+ 478us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:27:44.214302 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:27:44.214374 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:44.214520 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 23:27:44.214596 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 23:27:44.215090 21864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:44.208744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:43007 (local address 127.9.44.65:48093)
0808 23:27:44.209391 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:44.209409 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:44.209603 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:44.210112 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:44.210121 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:44.210135 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 23:27:44.210438 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:44.210449 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.211533 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.211538 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:44.212987 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:44.213001 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.213854 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.213859 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:44.213989 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:44.214047 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 23:27:44.214405 (+ 358us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 23:27:44.214416 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 23:27:44.214496 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 23:27:44.214623 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":619,"threads_started":1}
I20260808 23:27:44.243955 20870 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:27:44.244177 20870 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:27:44.246863 20870 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167: Sending DeleteTablet for 3 replicas of tablet 6ac206fbb47a4e709f04f194d20c9f82
I20260808 23:27:44.247493 21120 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ac206fbb47a4e709f04f194d20c9f82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:27:44 UTC) from {username='oryx', principal='oryx/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:40124
I20260808 23:27:44.247689 21351 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ac206fbb47a4e709f04f194d20c9f82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:27:44 UTC) from {username='oryx', principal='oryx/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:52842
I20260808 23:27:44.248117 21594 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ac206fbb47a4e709f04f194d20c9f82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:27:44 UTC) from {username='oryx', principal='oryx/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:52946
I20260808 23:27:44.259949 21869 tablet_replica.cc:330] T 6ac206fbb47a4e709f04f194d20c9f82 P d0341bb2dab042899a597a550f919011: stopping tablet replica
I20260808 23:27:44.260073 21868 tablet_replica.cc:330] T 6ac206fbb47a4e709f04f194d20c9f82 P adec41244a7d4304aba0dcac828d454d: stopping tablet replica
I20260808 23:27:44.260159 21869 raft_consensus.cc:2229] T 6ac206fbb47a4e709f04f194d20c9f82 P d0341bb2dab042899a597a550f919011 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:27:44.260251 21868 raft_consensus.cc:2229] T 6ac206fbb47a4e709f04f194d20c9f82 P adec41244a7d4304aba0dcac828d454d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:27:44.260434 21869 raft_consensus.cc:2258] T 6ac206fbb47a4e709f04f194d20c9f82 P d0341bb2dab042899a597a550f919011 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:27:44.260505 21868 raft_consensus.cc:2258] T 6ac206fbb47a4e709f04f194d20c9f82 P adec41244a7d4304aba0dcac828d454d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:27:44.261041 21870 tablet_replica.cc:330] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72: stopping tablet replica
I20260808 23:27:44.261230 21870 raft_consensus.cc:2229] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:27:44.261587 21870 raft_consensus.cc:2258] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:27:44.261644 21868 ts_tablet_manager.cc:1905] T 6ac206fbb47a4e709f04f194d20c9f82 P adec41244a7d4304aba0dcac828d454d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:27:44.261780 21869 ts_tablet_manager.cc:1905] T 6ac206fbb47a4e709f04f194d20c9f82 P d0341bb2dab042899a597a550f919011: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:27:44.264770 21870 ts_tablet_manager.cc:1905] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:27:44.268807 21869 ts_tablet_manager.cc:1918] T 6ac206fbb47a4e709f04f194d20c9f82 P d0341bb2dab042899a597a550f919011: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:27:44.268998 21869 log.cc:1196] T 6ac206fbb47a4e709f04f194d20c9f82 P d0341bb2dab042899a597a550f919011: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/6ac206fbb47a4e709f04f194d20c9f82
I20260808 23:27:44.269367 21868 ts_tablet_manager.cc:1918] T 6ac206fbb47a4e709f04f194d20c9f82 P adec41244a7d4304aba0dcac828d454d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:27:44.269502 21868 log.cc:1196] T 6ac206fbb47a4e709f04f194d20c9f82 P adec41244a7d4304aba0dcac828d454d: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-0/wal/wals/6ac206fbb47a4e709f04f194d20c9f82
I20260808 23:27:44.269668 21869 ts_tablet_manager.cc:1939] T 6ac206fbb47a4e709f04f194d20c9f82 P d0341bb2dab042899a597a550f919011: Deleting consensus metadata
I20260808 23:27:44.269961 21868 ts_tablet_manager.cc:1939] T 6ac206fbb47a4e709f04f194d20c9f82 P adec41244a7d4304aba0dcac828d454d: Deleting consensus metadata
I20260808 23:27:44.270893 20846 catalog_manager.cc:4841] TS d0341bb2dab042899a597a550f919011 (127.9.44.66:45343): tablet 6ac206fbb47a4e709f04f194d20c9f82 (table test-table [id=7bf582f44ab64f7787c00dc285cfa3b9]) successfully deleted
I20260808 23:27:44.271140 21870 ts_tablet_manager.cc:1918] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:27:44.271148 20846 catalog_manager.cc:4841] TS adec41244a7d4304aba0dcac828d454d (127.9.44.65:43811): tablet 6ac206fbb47a4e709f04f194d20c9f82 (table test-table [id=7bf582f44ab64f7787c00dc285cfa3b9]) successfully deleted
I20260808 23:27:44.271226 21870 log.cc:1196] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/6ac206fbb47a4e709f04f194d20c9f82
I20260808 23:27:44.271512 21870 ts_tablet_manager.cc:1939] T 6ac206fbb47a4e709f04f194d20c9f82 P e3c4f5915fcc462194b8ee854a3bee72: Deleting consensus metadata
I20260808 23:27:44.272423 20847 catalog_manager.cc:4841] TS e3c4f5915fcc462194b8ee854a3bee72 (127.9.44.67:43007): tablet 6ac206fbb47a4e709f04f194d20c9f82 (table test-table [id=7bf582f44ab64f7787c00dc285cfa3b9]) successfully deleted
Aug 08 23:27:44 dist-test-slave-6bbx krb5kdc[20787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 23:27:44 dist-test-slave-6bbx krb5kdc[20787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:44 dist-test-slave-6bbx krb5kdc[20787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 23:27:44.319214 21881 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:44.309147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58670 (local address 127.9.44.126:43315)
0808 23:27:44.309981 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:44.309985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:44.310002 (+ 17us) server_negotiation.cc:407] Connection header received
0808 23:27:44.310055 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:44.310058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:44.310120 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:44.310224 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:44.311617 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.312549 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.313818 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.314219 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.314339 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:44.318914 (+ 4575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58670: BlockingRecv error: recv got EOF from 127.0.0.1:58670 (error 108)
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":539,"threads_started":1}
Aug 08 23:27:44 dist-test-slave-6bbx krb5kdc[20787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM
I20260808 23:27:44.338164 21881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:44.323279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58672 (local address 127.9.44.126:43315)
0808 23:27:44.323446 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:44.323451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:44.323766 (+ 315us) server_negotiation.cc:407] Connection header received
0808 23:27:44.323933 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:44.323937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:44.323993 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:44.324060 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:44.325491 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.326179 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.328002 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.328398 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.328486 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:44.332583 (+ 4097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:44.332587 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:44.332590 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:44.332621 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:44.335799 (+ 3178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:44.336406 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:44.336411 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:44.336414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:44.336570 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:44.336885 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:44.336890 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:44.336893 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:44.337219 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:44.337464 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:44.337636 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 23:27:44.337856 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 23:27:44.341392 20870 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58672:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:27:44.342015 20870 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:27:44.348798 21120 tablet_service.cc:1467] Processing CreateTablet for tablet 4aca5bb59d4b4666b9ec6dd32ffc357f (DEFAULT_TABLE table=test-table [id=a0bce3f86450413e8c3babc15aa743cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:27:44.349321 21120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aca5bb59d4b4666b9ec6dd32ffc357f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:44.349200 21594 tablet_service.cc:1467] Processing CreateTablet for tablet 4aca5bb59d4b4666b9ec6dd32ffc357f (DEFAULT_TABLE table=test-table [id=a0bce3f86450413e8c3babc15aa743cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:27:44.349642 21594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aca5bb59d4b4666b9ec6dd32ffc357f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:44.349727 21351 tablet_service.cc:1467] Processing CreateTablet for tablet 4aca5bb59d4b4666b9ec6dd32ffc357f (DEFAULT_TABLE table=test-table [id=a0bce3f86450413e8c3babc15aa743cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:27:44.350162 21351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aca5bb59d4b4666b9ec6dd32ffc357f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:44.360970 21891 tablet_bootstrap.cc:492] T 4aca5bb59d4b4666b9ec6dd32ffc357f P e3c4f5915fcc462194b8ee854a3bee72: Bootstrap starting.
I20260808 23:27:44.362944 21891 tablet_bootstrap.cc:654] T 4aca5bb59d4b4666b9ec6dd32ffc357f P e3c4f5915fcc462194b8ee854a3bee72: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:44.364344 21892 tablet_bootstrap.cc:492] T 4aca5bb59d4b4666b9ec6dd32ffc357f P adec41244a7d4304aba0dcac828d454d: Bootstrap starting.
I20260808 23:27:44.365506 21894 tablet_bootstrap.cc:492] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011: Bootstrap starting.
I20260808 23:27:44.366426 21892 tablet_bootstrap.cc:654] T 4aca5bb59d4b4666b9ec6dd32ffc357f P adec41244a7d4304aba0dcac828d454d: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:44.367156 21894 tablet_bootstrap.cc:654] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:44.370330 21891 tablet_bootstrap.cc:492] T 4aca5bb59d4b4666b9ec6dd32ffc357f P e3c4f5915fcc462194b8ee854a3bee72: No bootstrap required, opened a new log
I20260808 23:27:44.370337 21892 tablet_bootstrap.cc:492] T 4aca5bb59d4b4666b9ec6dd32ffc357f P adec41244a7d4304aba0dcac828d454d: No bootstrap required, opened a new log
I20260808 23:27:44.370478 21892 ts_tablet_manager.cc:1397] T 4aca5bb59d4b4666b9ec6dd32ffc357f P adec41244a7d4304aba0dcac828d454d: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.002s
I20260808 23:27:44.370479 21891 ts_tablet_manager.cc:1397] T 4aca5bb59d4b4666b9ec6dd32ffc357f P e3c4f5915fcc462194b8ee854a3bee72: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.002s
I20260808 23:27:44.370617 21894 tablet_bootstrap.cc:492] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011: No bootstrap required, opened a new log
I20260808 23:27:44.370746 21894 ts_tablet_manager.cc:1397] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260808 23:27:44.371248 21891 raft_consensus.cc:348] T 4aca5bb59d4b4666b9ec6dd32ffc357f P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:44.371305 21892 raft_consensus.cc:348] T 4aca5bb59d4b4666b9ec6dd32ffc357f P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:44.371512 21891 raft_consensus.cc:374] T 4aca5bb59d4b4666b9ec6dd32ffc357f P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:44.371587 21892 raft_consensus.cc:374] T 4aca5bb59d4b4666b9ec6dd32ffc357f P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:44.371598 21891 raft_consensus.cc:729] T 4aca5bb59d4b4666b9ec6dd32ffc357f P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3c4f5915fcc462194b8ee854a3bee72, State: Initialized, Role: FOLLOWER
I20260808 23:27:44.371714 21892 raft_consensus.cc:729] T 4aca5bb59d4b4666b9ec6dd32ffc357f P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adec41244a7d4304aba0dcac828d454d, State: Initialized, Role: FOLLOWER
I20260808 23:27:44.371958 21892 consensus_queue.cc:260] T 4aca5bb59d4b4666b9ec6dd32ffc357f P adec41244a7d4304aba0dcac828d454d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:44.372611 21892 ts_tablet_manager.cc:1428] T 4aca5bb59d4b4666b9ec6dd32ffc357f P adec41244a7d4304aba0dcac828d454d: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260808 23:27:44.371439 21894 raft_consensus.cc:348] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:44.372759 21894 raft_consensus.cc:374] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:44.372869 21894 raft_consensus.cc:729] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0341bb2dab042899a597a550f919011, State: Initialized, Role: FOLLOWER
I20260808 23:27:44.373080 21894 consensus_queue.cc:260] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:44.374518 21894 ts_tablet_manager.cc:1428] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260808 23:27:44.374737 21891 consensus_queue.cc:260] T 4aca5bb59d4b4666b9ec6dd32ffc357f P e3c4f5915fcc462194b8ee854a3bee72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:44.376251 21891 ts_tablet_manager.cc:1428] T 4aca5bb59d4b4666b9ec6dd32ffc357f P e3c4f5915fcc462194b8ee854a3bee72: Time spent starting tablet: real 0.006s user 0.001s sys 0.002s
I20260808 23:27:44.591886 21748 raft_consensus.cc:482] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:27:44.592087 21748 raft_consensus.cc:504] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:44.593487 21748 leader_election.cc:288] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adec41244a7d4304aba0dcac828d454d (127.9.44.65:43811), e3c4f5915fcc462194b8ee854a3bee72 (127.9.44.67:43007)
I20260808 23:27:44.599921 21820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:44.593988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:49739 (local address 127.9.44.67:43007)
0808 23:27:44.594163 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:44.594169 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:44.594628 (+ 459us) server_negotiation.cc:407] Connection header received
0808 23:27:44.594784 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:44.594788 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:44.594847 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:44.594925 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:44.595582 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.596382 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.598266 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.599019 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.599129 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:44.599241 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:44.599506 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 23:27:44.599631 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 23:27:44.600029 21903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:44.593799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:43007 (local address 127.9.44.66:49739)
0808 23:27:44.594523 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:44.594537 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:44.594657 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:44.594984 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:44.594988 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:44.595029 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:44.595427 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:44.595439 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.596512 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.596517 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:44.598132 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:44.598146 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.599165 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.599171 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:44.599349 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:44.599371 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 23:27:44.599455 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 23:27:44.599578 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":21,"thread_start_us":425,"threads_started":1}
I20260808 23:27:44.600109 21839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:44.593778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:52321 (local address 127.9.44.65:43811)
0808 23:27:44.593984 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:44.593992 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:44.594548 (+ 556us) server_negotiation.cc:407] Connection header received
0808 23:27:44.594720 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:44.594724 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:44.594794 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:44.594877 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:27:44.595502 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.596293 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.598300 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.599087 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.599223 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:44.599303 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:44.599732 (+ 429us) server_negotiation.cc:299] Negotiation successful
0808 23:27:44.599865 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 23:27:44.600185 21902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:44.593716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:43811 (local address 127.9.44.66:52321)
0808 23:27:44.594445 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:44.594461 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:44.594591 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:44.594917 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:44.594921 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:44.594960 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:27:44.595302 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:44.595320 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.596435 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.596440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:44.598148 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:44.598160 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.599433 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.599437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:44.599558 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:44.599580 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 23:27:44.599658 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 23:27:44.599803 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"mutex_wait_us":90,"thread_start_us":433,"threads_started":1}
I20260808 23:27:44.600631 21631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aca5bb59d4b4666b9ec6dd32ffc357f" candidate_uuid: "d0341bb2dab042899a597a550f919011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3c4f5915fcc462194b8ee854a3bee72" is_pre_election: true
I20260808 23:27:44.600901 21631 raft_consensus.cc:2454] T 4aca5bb59d4b4666b9ec6dd32ffc357f P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0341bb2dab042899a597a550f919011 in term 0.
I20260808 23:27:44.600996 21154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aca5bb59d4b4666b9ec6dd32ffc357f" candidate_uuid: "d0341bb2dab042899a597a550f919011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adec41244a7d4304aba0dcac828d454d" is_pre_election: true
I20260808 23:27:44.601257 21154 raft_consensus.cc:2454] T 4aca5bb59d4b4666b9ec6dd32ffc357f P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0341bb2dab042899a597a550f919011 in term 0.
I20260808 23:27:44.601338 21273 leader_election.cc:302] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0341bb2dab042899a597a550f919011, e3c4f5915fcc462194b8ee854a3bee72; no voters:
I20260808 23:27:44.601778 21748 raft_consensus.cc:2781] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:27:44.601907 21748 raft_consensus.cc:482] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:27:44.602007 21748 raft_consensus.cc:3037] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:44.604126 21748 raft_consensus.cc:504] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:44.604600 21748 leader_election.cc:288] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011 [CANDIDATE]: Term 1 election: Requested vote from peers adec41244a7d4304aba0dcac828d454d (127.9.44.65:43811), e3c4f5915fcc462194b8ee854a3bee72 (127.9.44.67:43007)
I20260808 23:27:44.605015 21154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aca5bb59d4b4666b9ec6dd32ffc357f" candidate_uuid: "d0341bb2dab042899a597a550f919011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adec41244a7d4304aba0dcac828d454d"
I20260808 23:27:44.605084 21631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aca5bb59d4b4666b9ec6dd32ffc357f" candidate_uuid: "d0341bb2dab042899a597a550f919011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3c4f5915fcc462194b8ee854a3bee72"
I20260808 23:27:44.605242 21154 raft_consensus.cc:3037] T 4aca5bb59d4b4666b9ec6dd32ffc357f P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:44.605262 21631 raft_consensus.cc:3037] T 4aca5bb59d4b4666b9ec6dd32ffc357f P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:44.608028 21154 raft_consensus.cc:2454] T 4aca5bb59d4b4666b9ec6dd32ffc357f P adec41244a7d4304aba0dcac828d454d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0341bb2dab042899a597a550f919011 in term 1.
I20260808 23:27:44.608042 21631 raft_consensus.cc:2454] T 4aca5bb59d4b4666b9ec6dd32ffc357f P e3c4f5915fcc462194b8ee854a3bee72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0341bb2dab042899a597a550f919011 in term 1.
I20260808 23:27:44.608417 21270 leader_election.cc:302] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adec41244a7d4304aba0dcac828d454d, d0341bb2dab042899a597a550f919011; no voters:
I20260808 23:27:44.608697 21748 raft_consensus.cc:2781] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:27:44.609052 21748 raft_consensus.cc:686] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011 [term 1 LEADER]: Becoming Leader. State: Replica: d0341bb2dab042899a597a550f919011, State: Running, Role: LEADER
I20260808 23:27:44.609405 21748 consensus_queue.cc:237] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:44.612600 20870 catalog_manager.cc:5495] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011 reported cstate change: term changed from 0 to 1, leader changed from <none> to d0341bb2dab042899a597a550f919011 (127.9.44.66). New cstate: current_term: 1 leader_uuid: "d0341bb2dab042899a597a550f919011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } health_report { overall_health: UNKNOWN } } }
I20260808 23:27:44.656980 21910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:44.651425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52858 (local address 127.9.44.66:45343)
0808 23:27:44.652021 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:44.652025 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:44.652043 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:27:44.652115 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:44.652118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:44.652184 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:44.652328 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:27:44.652743 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.653498 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.654765 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.655198 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.655305 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:44.655792 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:27:44.656354 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:27:44.656495 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:44.656610 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 23:27:44.656685 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":360,"threads_started":1}
I20260808 23:27:44.660223 21154 raft_consensus.cc:1261] T 4aca5bb59d4b4666b9ec6dd32ffc357f P adec41244a7d4304aba0dcac828d454d [term 1 FOLLOWER]: Refusing update from remote peer d0341bb2dab042899a597a550f919011: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:27:44.660213 21631 raft_consensus.cc:1261] T 4aca5bb59d4b4666b9ec6dd32ffc357f P e3c4f5915fcc462194b8ee854a3bee72 [term 1 FOLLOWER]: Refusing update from remote peer d0341bb2dab042899a597a550f919011: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:27:44.660912 21748 consensus_queue.cc:1035] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:27:44.661222 21906 consensus_queue.cc:1035] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:27:44.665592 21823 mvcc.cc:205] Tried to move back new op lower bound from 7316404898442129408 to 7316404898243366912. Current Snapshot: MvccSnapshot[applied={T|T < 7316404898442129408}]
I20260808 23:27:44.668733 21834 mvcc.cc:205] Tried to move back new op lower bound from 7316404898442129408 to 7316404898243366912. Current Snapshot: MvccSnapshot[applied={T|T < 7316404898442129408 or (T in {7316404898442129408})}]
I20260808 23:27:44.676887 20870 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:27:44.677138 20870 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:27:44.679889 20870 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167: Sending DeleteTablet for 3 replicas of tablet 4aca5bb59d4b4666b9ec6dd32ffc357f
I20260808 23:27:44.680697 21594 tablet_service.cc:1514] Processing DeleteTablet for tablet 4aca5bb59d4b4666b9ec6dd32ffc357f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:27:44 UTC) from {username='oryx', principal='oryx/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:52946
I20260808 23:27:44.680728 21120 tablet_service.cc:1514] Processing DeleteTablet for tablet 4aca5bb59d4b4666b9ec6dd32ffc357f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:27:44 UTC) from {username='oryx', principal='oryx/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:40124
I20260808 23:27:44.680969 21870 tablet_replica.cc:330] T 4aca5bb59d4b4666b9ec6dd32ffc357f P e3c4f5915fcc462194b8ee854a3bee72: stopping tablet replica
I20260808 23:27:44.680984 21868 tablet_replica.cc:330] T 4aca5bb59d4b4666b9ec6dd32ffc357f P adec41244a7d4304aba0dcac828d454d: stopping tablet replica
I20260808 23:27:44.681015 21351 tablet_service.cc:1514] Processing DeleteTablet for tablet 4aca5bb59d4b4666b9ec6dd32ffc357f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:27:44 UTC) from {username='oryx', principal='oryx/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:52842
I20260808 23:27:44.681164 21868 raft_consensus.cc:2229] T 4aca5bb59d4b4666b9ec6dd32ffc357f P adec41244a7d4304aba0dcac828d454d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:27:44.681185 21870 raft_consensus.cc:2229] T 4aca5bb59d4b4666b9ec6dd32ffc357f P e3c4f5915fcc462194b8ee854a3bee72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:27:44.681358 21869 tablet_replica.cc:330] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011: stopping tablet replica
I20260808 23:27:44.681370 21868 raft_consensus.cc:2258] T 4aca5bb59d4b4666b9ec6dd32ffc357f P adec41244a7d4304aba0dcac828d454d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:27:44.681427 21870 raft_consensus.cc:2258] T 4aca5bb59d4b4666b9ec6dd32ffc357f P e3c4f5915fcc462194b8ee854a3bee72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:27:44.681542 21869 raft_consensus.cc:2229] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:27:44.682039 21869 raft_consensus.cc:2258] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:27:44.682346 21868 ts_tablet_manager.cc:1905] T 4aca5bb59d4b4666b9ec6dd32ffc357f P adec41244a7d4304aba0dcac828d454d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:27:44.683012 21870 ts_tablet_manager.cc:1905] T 4aca5bb59d4b4666b9ec6dd32ffc357f P e3c4f5915fcc462194b8ee854a3bee72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:27:44.683094 20870 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58672:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:27:44.683710 20870 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:27:44.688347 21869 ts_tablet_manager.cc:1905] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:27:44.688637 21868 ts_tablet_manager.cc:1918] T 4aca5bb59d4b4666b9ec6dd32ffc357f P adec41244a7d4304aba0dcac828d454d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:27:44.688777 21868 log.cc:1196] T 4aca5bb59d4b4666b9ec6dd32ffc357f P adec41244a7d4304aba0dcac828d454d: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-0/wal/wals/4aca5bb59d4b4666b9ec6dd32ffc357f
I20260808 23:27:44.690757 21870 ts_tablet_manager.cc:1918] T 4aca5bb59d4b4666b9ec6dd32ffc357f P e3c4f5915fcc462194b8ee854a3bee72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:27:44.690973 21870 log.cc:1196] T 4aca5bb59d4b4666b9ec6dd32ffc357f P e3c4f5915fcc462194b8ee854a3bee72: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/4aca5bb59d4b4666b9ec6dd32ffc357f
I20260808 23:27:44.691429 21870 ts_tablet_manager.cc:1939] T 4aca5bb59d4b4666b9ec6dd32ffc357f P e3c4f5915fcc462194b8ee854a3bee72: Deleting consensus metadata
I20260808 23:27:44.692530 20847 catalog_manager.cc:4841] TS e3c4f5915fcc462194b8ee854a3bee72 (127.9.44.67:43007): tablet 4aca5bb59d4b4666b9ec6dd32ffc357f (table test-table [id=a0bce3f86450413e8c3babc15aa743cf]) successfully deleted
I20260808 23:27:44.693522 21868 ts_tablet_manager.cc:1939] T 4aca5bb59d4b4666b9ec6dd32ffc357f P adec41244a7d4304aba0dcac828d454d: Deleting consensus metadata
I20260808 23:27:44.694340 21351 tablet_service.cc:1467] Processing CreateTablet for tablet 007cca3c0a304aee979cfb7fac947fe5 (DEFAULT_TABLE table=test-table [id=110d05c349b24e2692a80c6c9857d56f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:27:44.694759 21351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 007cca3c0a304aee979cfb7fac947fe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:44.696038 21594 tablet_service.cc:1467] Processing CreateTablet for tablet 007cca3c0a304aee979cfb7fac947fe5 (DEFAULT_TABLE table=test-table [id=110d05c349b24e2692a80c6c9857d56f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:27:44.696470 21594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 007cca3c0a304aee979cfb7fac947fe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:44.696916 21120 tablet_service.cc:1467] Processing CreateTablet for tablet 007cca3c0a304aee979cfb7fac947fe5 (DEFAULT_TABLE table=test-table [id=110d05c349b24e2692a80c6c9857d56f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:27:44.697304 21120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 007cca3c0a304aee979cfb7fac947fe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:27:44.699898 20846 catalog_manager.cc:4841] TS adec41244a7d4304aba0dcac828d454d (127.9.44.65:43811): tablet 4aca5bb59d4b4666b9ec6dd32ffc357f (table test-table [id=a0bce3f86450413e8c3babc15aa743cf]) successfully deleted
I20260808 23:27:44.701009 21894 tablet_bootstrap.cc:492] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011: Bootstrap starting.
I20260808 23:27:44.702696 21894 tablet_bootstrap.cc:654] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:44.702940 21891 tablet_bootstrap.cc:492] T 007cca3c0a304aee979cfb7fac947fe5 P e3c4f5915fcc462194b8ee854a3bee72: Bootstrap starting.
I20260808 23:27:44.704052 21892 tablet_bootstrap.cc:492] T 007cca3c0a304aee979cfb7fac947fe5 P adec41244a7d4304aba0dcac828d454d: Bootstrap starting.
I20260808 23:27:44.704464 21869 ts_tablet_manager.cc:1918] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:27:44.704517 21891 tablet_bootstrap.cc:654] T 007cca3c0a304aee979cfb7fac947fe5 P e3c4f5915fcc462194b8ee854a3bee72: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:44.704625 21869 log.cc:1196] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/4aca5bb59d4b4666b9ec6dd32ffc357f
I20260808 23:27:44.705080 21869 ts_tablet_manager.cc:1939] T 4aca5bb59d4b4666b9ec6dd32ffc357f P d0341bb2dab042899a597a550f919011: Deleting consensus metadata
I20260808 23:27:44.705524 21892 tablet_bootstrap.cc:654] T 007cca3c0a304aee979cfb7fac947fe5 P adec41244a7d4304aba0dcac828d454d: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:44.706205 21894 tablet_bootstrap.cc:492] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011: No bootstrap required, opened a new log
I20260808 23:27:44.706430 21894 ts_tablet_manager.cc:1397] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260808 23:27:44.707317 20846 catalog_manager.cc:4841] TS d0341bb2dab042899a597a550f919011 (127.9.44.66:45343): tablet 4aca5bb59d4b4666b9ec6dd32ffc357f (table test-table [id=a0bce3f86450413e8c3babc15aa743cf]) successfully deleted
I20260808 23:27:44.707263 21894 raft_consensus.cc:348] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:44.707741 21894 raft_consensus.cc:374] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:44.707873 21894 raft_consensus.cc:729] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0341bb2dab042899a597a550f919011, State: Initialized, Role: FOLLOWER
I20260808 23:27:44.708177 21894 consensus_queue.cc:260] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:44.708783 21894 ts_tablet_manager.cc:1428] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 23:27:44.709970 21892 tablet_bootstrap.cc:492] T 007cca3c0a304aee979cfb7fac947fe5 P adec41244a7d4304aba0dcac828d454d: No bootstrap required, opened a new log
I20260808 23:27:44.710137 21892 ts_tablet_manager.cc:1397] T 007cca3c0a304aee979cfb7fac947fe5 P adec41244a7d4304aba0dcac828d454d: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260808 23:27:44.710820 21892 raft_consensus.cc:348] T 007cca3c0a304aee979cfb7fac947fe5 P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:44.711004 21892 raft_consensus.cc:374] T 007cca3c0a304aee979cfb7fac947fe5 P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:44.711087 21892 raft_consensus.cc:729] T 007cca3c0a304aee979cfb7fac947fe5 P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adec41244a7d4304aba0dcac828d454d, State: Initialized, Role: FOLLOWER
I20260808 23:27:44.711318 21892 consensus_queue.cc:260] T 007cca3c0a304aee979cfb7fac947fe5 P adec41244a7d4304aba0dcac828d454d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:44.713318 21892 ts_tablet_manager.cc:1428] T 007cca3c0a304aee979cfb7fac947fe5 P adec41244a7d4304aba0dcac828d454d: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260808 23:27:44.718880 21891 tablet_bootstrap.cc:492] T 007cca3c0a304aee979cfb7fac947fe5 P e3c4f5915fcc462194b8ee854a3bee72: No bootstrap required, opened a new log
I20260808 23:27:44.719033 21891 ts_tablet_manager.cc:1397] T 007cca3c0a304aee979cfb7fac947fe5 P e3c4f5915fcc462194b8ee854a3bee72: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.008s
I20260808 23:27:44.719642 21891 raft_consensus.cc:348] T 007cca3c0a304aee979cfb7fac947fe5 P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:44.719892 21891 raft_consensus.cc:374] T 007cca3c0a304aee979cfb7fac947fe5 P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:44.719964 21891 raft_consensus.cc:729] T 007cca3c0a304aee979cfb7fac947fe5 P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3c4f5915fcc462194b8ee854a3bee72, State: Initialized, Role: FOLLOWER
I20260808 23:27:44.720223 21891 consensus_queue.cc:260] T 007cca3c0a304aee979cfb7fac947fe5 P e3c4f5915fcc462194b8ee854a3bee72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:44.720676 21891 ts_tablet_manager.cc:1428] T 007cca3c0a304aee979cfb7fac947fe5 P e3c4f5915fcc462194b8ee854a3bee72: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260808 23:27:44.869203 21906 raft_consensus.cc:482] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:27:44.869432 21906 raft_consensus.cc:504] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:44.870236 21906 leader_election.cc:288] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adec41244a7d4304aba0dcac828d454d (127.9.44.65:43811), e3c4f5915fcc462194b8ee854a3bee72 (127.9.44.67:43007)
I20260808 23:27:44.870756 21631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "007cca3c0a304aee979cfb7fac947fe5" candidate_uuid: "d0341bb2dab042899a597a550f919011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3c4f5915fcc462194b8ee854a3bee72" is_pre_election: true
I20260808 23:27:44.870784 21154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "007cca3c0a304aee979cfb7fac947fe5" candidate_uuid: "d0341bb2dab042899a597a550f919011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adec41244a7d4304aba0dcac828d454d" is_pre_election: true
I20260808 23:27:44.870988 21631 raft_consensus.cc:2454] T 007cca3c0a304aee979cfb7fac947fe5 P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0341bb2dab042899a597a550f919011 in term 0.
I20260808 23:27:44.871047 21154 raft_consensus.cc:2454] T 007cca3c0a304aee979cfb7fac947fe5 P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0341bb2dab042899a597a550f919011 in term 0.
I20260808 23:27:44.871426 21273 leader_election.cc:302] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0341bb2dab042899a597a550f919011, e3c4f5915fcc462194b8ee854a3bee72; no voters:
I20260808 23:27:44.871744 21906 raft_consensus.cc:2781] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:27:44.871842 21906 raft_consensus.cc:482] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:27:44.871961 21906 raft_consensus.cc:3037] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:44.874223 21906 raft_consensus.cc:504] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:44.874770 21906 leader_election.cc:288] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011 [CANDIDATE]: Term 1 election: Requested vote from peers adec41244a7d4304aba0dcac828d454d (127.9.44.65:43811), e3c4f5915fcc462194b8ee854a3bee72 (127.9.44.67:43007)
I20260808 23:27:44.875172 21154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "007cca3c0a304aee979cfb7fac947fe5" candidate_uuid: "d0341bb2dab042899a597a550f919011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adec41244a7d4304aba0dcac828d454d"
I20260808 23:27:44.875280 21631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "007cca3c0a304aee979cfb7fac947fe5" candidate_uuid: "d0341bb2dab042899a597a550f919011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3c4f5915fcc462194b8ee854a3bee72"
I20260808 23:27:44.875407 21154 raft_consensus.cc:3037] T 007cca3c0a304aee979cfb7fac947fe5 P adec41244a7d4304aba0dcac828d454d [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:44.875469 21631 raft_consensus.cc:3037] T 007cca3c0a304aee979cfb7fac947fe5 P e3c4f5915fcc462194b8ee854a3bee72 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:44.878170 21154 raft_consensus.cc:2454] T 007cca3c0a304aee979cfb7fac947fe5 P adec41244a7d4304aba0dcac828d454d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0341bb2dab042899a597a550f919011 in term 1.
I20260808 23:27:44.878185 21631 raft_consensus.cc:2454] T 007cca3c0a304aee979cfb7fac947fe5 P e3c4f5915fcc462194b8ee854a3bee72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0341bb2dab042899a597a550f919011 in term 1.
I20260808 23:27:44.878566 21273 leader_election.cc:302] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0341bb2dab042899a597a550f919011, e3c4f5915fcc462194b8ee854a3bee72; no voters:
I20260808 23:27:44.878819 21906 raft_consensus.cc:2781] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:27:44.879001 21906 raft_consensus.cc:686] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011 [term 1 LEADER]: Becoming Leader. State: Replica: d0341bb2dab042899a597a550f919011, State: Running, Role: LEADER
I20260808 23:27:44.879247 21906 consensus_queue.cc:237] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } }
I20260808 23:27:44.881937 20866 catalog_manager.cc:5495] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011 reported cstate change: term changed from 0 to 1, leader changed from <none> to d0341bb2dab042899a597a550f919011 (127.9.44.66). New cstate: current_term: 1 leader_uuid: "d0341bb2dab042899a597a550f919011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0341bb2dab042899a597a550f919011" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45343 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 } health_report { overall_health: UNKNOWN } } }
I20260808 23:27:44.919658 21839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:44.913838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:39789 (local address 127.9.44.65:43811)
0808 23:27:44.914023 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:44.914030 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:44.914647 (+ 617us) server_negotiation.cc:407] Connection header received
0808 23:27:44.914783 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:44.914788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:44.914875 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:44.914975 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:27:44.915464 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.916534 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.917994 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.918402 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.918511 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:44.918973 (+ 462us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:27:44.919079 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:27:44.919152 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:44.919314 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 23:27:44.919390 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 23:27:44.919917 21926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:44.913850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:43811 (local address 127.9.44.66:39789)
0808 23:27:44.914541 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:44.914557 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:44.914668 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:44.914976 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:44.914979 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:44.914987 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 23:27:44.915296 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:44.915307 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.916658 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.916662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:44.917857 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:44.917870 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.918582 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.918587 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:44.918750 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:44.918807 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 23:27:44.919178 (+ 371us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 23:27:44.919190 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 23:27:44.919280 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 23:27:44.919415 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":388,"threads_started":1}
I20260808 23:27:44.927964 21631 raft_consensus.cc:1261] T 007cca3c0a304aee979cfb7fac947fe5 P e3c4f5915fcc462194b8ee854a3bee72 [term 1 FOLLOWER]: Refusing update from remote peer d0341bb2dab042899a597a550f919011: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:27:44.927990 21154 raft_consensus.cc:1261] T 007cca3c0a304aee979cfb7fac947fe5 P adec41244a7d4304aba0dcac828d454d [term 1 FOLLOWER]: Refusing update from remote peer d0341bb2dab042899a597a550f919011: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:27:44.928546 21906 consensus_queue.cc:1035] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3c4f5915fcc462194b8ee854a3bee72" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 43007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:27:44.928965 21906 consensus_queue.cc:1035] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adec41244a7d4304aba0dcac828d454d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:27:44.981998 21910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:44.973128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:53147 (local address 127.9.44.66:45343)
0808 23:27:44.973294 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:44.973300 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:44.976497 (+ 3197us) server_negotiation.cc:407] Connection header received
0808 23:27:44.976776 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:44.976781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:44.976863 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:44.976932 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:27:44.977637 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.978617 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.980260 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.980762 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.980875 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:44.981217 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:27:44.981319 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:27:44.981389 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:44.981588 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 23:27:44.981660 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 23:27:44.982021 21942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:44.972766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:45343 (local address 127.9.44.65:53147)
0808 23:27:44.976400 (+ 3634us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:44.976419 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:44.976608 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:44.977162 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:44.977167 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:44.977180 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 23:27:44.977471 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:44.977483 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.978763 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.978767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:44.980129 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:44.980142 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:44.980905 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:44.980911 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:44.981040 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:44.981077 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 23:27:44.981411 (+ 334us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 23:27:44.981423 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 23:27:44.981557 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 23:27:44.981689 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3482,"thread_start_us":3335,"threads_started":1}
I20260808 23:27:45.040778 20870 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:27:45.041050 20870 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:27:45.044550 20870 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72837814baea4248ad80b660e43c5167: Sending DeleteTablet for 3 replicas of tablet 007cca3c0a304aee979cfb7fac947fe5
I20260808 23:27:45.045285 21120 tablet_service.cc:1514] Processing DeleteTablet for tablet 007cca3c0a304aee979cfb7fac947fe5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:27:45 UTC) from {username='oryx', principal='oryx/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:40124
I20260808 23:27:45.045584 21868 tablet_replica.cc:330] T 007cca3c0a304aee979cfb7fac947fe5 P adec41244a7d4304aba0dcac828d454d: stopping tablet replica
I20260808 23:27:45.045593 21351 tablet_service.cc:1514] Processing DeleteTablet for tablet 007cca3c0a304aee979cfb7fac947fe5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:27:45 UTC) from {username='oryx', principal='oryx/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:52842
I20260808 23:27:45.045599 21594 tablet_service.cc:1514] Processing DeleteTablet for tablet 007cca3c0a304aee979cfb7fac947fe5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:27:45 UTC) from {username='oryx', principal='oryx/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:52946
I20260808 23:27:45.045760 21868 raft_consensus.cc:2229] T 007cca3c0a304aee979cfb7fac947fe5 P adec41244a7d4304aba0dcac828d454d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:27:45.045914 21870 tablet_replica.cc:330] T 007cca3c0a304aee979cfb7fac947fe5 P e3c4f5915fcc462194b8ee854a3bee72: stopping tablet replica
I20260808 23:27:45.045962 21868 raft_consensus.cc:2258] T 007cca3c0a304aee979cfb7fac947fe5 P adec41244a7d4304aba0dcac828d454d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:27:45.045975 21869 tablet_replica.cc:330] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011: stopping tablet replica
I20260808 23:27:45.046078 21870 raft_consensus.cc:2229] T 007cca3c0a304aee979cfb7fac947fe5 P e3c4f5915fcc462194b8ee854a3bee72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:27:45.046192 21869 raft_consensus.cc:2229] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:27:45.046271 21870 raft_consensus.cc:2258] T 007cca3c0a304aee979cfb7fac947fe5 P e3c4f5915fcc462194b8ee854a3bee72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:27:45.046550 21869 raft_consensus.cc:2258] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:27:45.047518 21870 ts_tablet_manager.cc:1905] T 007cca3c0a304aee979cfb7fac947fe5 P e3c4f5915fcc462194b8ee854a3bee72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:27:45.047585 21868 ts_tablet_manager.cc:1905] T 007cca3c0a304aee979cfb7fac947fe5 P adec41244a7d4304aba0dcac828d454d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:27:45.048491 21869 ts_tablet_manager.cc:1905] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:27:45.052529 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 21000
I20260808 23:27:45.055541 21868 ts_tablet_manager.cc:1918] T 007cca3c0a304aee979cfb7fac947fe5 P adec41244a7d4304aba0dcac828d454d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:27:45.055711 21868 log.cc:1196] T 007cca3c0a304aee979cfb7fac947fe5 P adec41244a7d4304aba0dcac828d454d: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-0/wal/wals/007cca3c0a304aee979cfb7fac947fe5
I20260808 23:27:45.056180 21868 ts_tablet_manager.cc:1939] T 007cca3c0a304aee979cfb7fac947fe5 P adec41244a7d4304aba0dcac828d454d: Deleting consensus metadata
I20260808 23:27:45.057274 20846 catalog_manager.cc:4841] TS adec41244a7d4304aba0dcac828d454d (127.9.44.65:43811): tablet 007cca3c0a304aee979cfb7fac947fe5 (table test-table [id=110d05c349b24e2692a80c6c9857d56f]) successfully deleted
I20260808 23:27:45.057824 21870 ts_tablet_manager.cc:1918] T 007cca3c0a304aee979cfb7fac947fe5 P e3c4f5915fcc462194b8ee854a3bee72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:27:45.057956 21870 log.cc:1196] T 007cca3c0a304aee979cfb7fac947fe5 P e3c4f5915fcc462194b8ee854a3bee72: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/007cca3c0a304aee979cfb7fac947fe5
I20260808 23:27:45.058175 21869 ts_tablet_manager.cc:1918] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 23:27:45.058337 21869 log.cc:1196] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/007cca3c0a304aee979cfb7fac947fe5
I20260808 23:27:45.058423 21870 ts_tablet_manager.cc:1939] T 007cca3c0a304aee979cfb7fac947fe5 P e3c4f5915fcc462194b8ee854a3bee72: Deleting consensus metadata
I20260808 23:27:45.058804 21869 ts_tablet_manager.cc:1939] T 007cca3c0a304aee979cfb7fac947fe5 P d0341bb2dab042899a597a550f919011: Deleting consensus metadata
I20260808 23:27:45.059394 20847 catalog_manager.cc:4841] TS e3c4f5915fcc462194b8ee854a3bee72 (127.9.44.67:43007): tablet 007cca3c0a304aee979cfb7fac947fe5 (table test-table [id=110d05c349b24e2692a80c6c9857d56f]) successfully deleted
I20260808 23:27:45.059849 20846 catalog_manager.cc:4841] TS d0341bb2dab042899a597a550f919011 (127.9.44.66:45343): tablet 007cca3c0a304aee979cfb7fac947fe5 (table test-table [id=110d05c349b24e2692a80c6c9857d56f]) successfully deleted
Aug 08 23:27:45 dist-test-slave-6bbx krb5kdc[20787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.44.65@KRBTEST.COM
I20260808 23:27:45.073774 21839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:45.053787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40136 (local address 127.9.44.65:43811)
0808 23:27:45.053985 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:45.053992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:45.060409 (+ 6417us) server_negotiation.cc:407] Connection header received
0808 23:27:45.060590 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:45.060595 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:45.060666 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:45.060756 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:45.061899 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:45.062606 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:45.063960 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:45.064387 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:45.064494 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:45.068456 (+ 3962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:45.068470 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:45.068486 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:45.068515 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:45.071224 (+ 2709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:45.071862 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:45.071869 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:45.071872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:45.071990 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:45.072344 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:45.072347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:45.072349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:45.072688 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:45.072962 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:45.073278 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 23:27:45.073494 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 23:27:45.074410 21224 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:45.203226 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 21000
I20260808 23:27:45.232784 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 21241
Aug 08 23:27:45 dist-test-slave-6bbx krb5kdc[20787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.44.66@KRBTEST.COM
I20260808 23:27:45.247052 21910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:45.233479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52870 (local address 127.9.44.66:45343)
0808 23:27:45.233699 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:45.233703 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:45.233743 (+ 40us) server_negotiation.cc:407] Connection header received
0808 23:27:45.233909 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:45.233915 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:45.233999 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:45.234087 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:45.235201 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:45.235974 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:45.237211 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:45.237613 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:45.237703 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:45.241545 (+ 3842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:45.241558 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:45.241573 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:45.241606 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:45.244360 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:45.245012 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:45.245020 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:45.245022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:45.245160 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:45.245517 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:45.245522 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:45.245525 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:45.245877 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:45.246160 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:45.246471 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 23:27:45.246716 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 23:27:45.247685 21467 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:45.369280 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 21241
I20260808 23:27:45.400991 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 21477
Aug 08 23:27:45 dist-test-slave-6bbx krb5kdc[20787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.44.67@KRBTEST.COM
I20260808 23:27:45.415798 21954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:45.401684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52954 (local address 127.9.44.67:43007)
0808 23:27:45.402187 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:45.402192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:45.402207 (+ 15us) server_negotiation.cc:407] Connection header received
0808 23:27:45.402259 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:45.402263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:45.402334 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:45.402441 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:45.403738 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:45.404457 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:45.405677 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:45.406085 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:45.406198 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:45.410613 (+ 4415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:45.410630 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:45.410648 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:45.410683 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:45.413481 (+ 2798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:45.414042 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:45.414048 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:45.414051 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:45.414202 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:45.414475 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:45.414478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:45.414480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:45.414822 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:45.415059 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:45.415237 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 23:27:45.415402 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":240,"threads_started":1}
I20260808 23:27:45.416515 21700 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:45.537424 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 21477
I20260808 23:27:45.565685 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 20811
I20260808 23:27:45.566823 20944 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:45.678114 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 20811
2026-08-08T23:27:45Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6144 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:27:45 dist-test-slave-6bbx krb5kdc[21963](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:27:45 dist-test-slave-6bbx krb5kdc[21963](info): set up 2 sockets
Aug 08 23:27:45 dist-test-slave-6bbx krb5kdc[21963](info): commencing operation
krb5kdc: starting...
W20260808 23:27:46.864156 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.006s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:27:47 dist-test-slave-6bbx krb5kdc[21963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:27:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:27:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:27:47.088053 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:46017
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:44451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:46017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:47.408097 21988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:47.408385 21988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:47.408531 21988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:47.415539 21988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:27:47.415656 21988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:47.415762 21988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:47.415840 21988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:27:47.415915 21988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:27:47.425432 21988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:44451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:46017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:46017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:47.426668 21988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:47.428023 21988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:47.439195 21999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:47.439263 22001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:47.439294 21997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:47.440021 21988 server_base.cc:1034] running on GCE node
I20260808 23:27:47.440835 21988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:47.442097 21988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:47.443308 21988 hybrid_clock.cc:648] HybridClock initialized: now 1786231667443264 us; error 32 us; skew 500 ppm
I20260808 23:27:47.443642 21988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T23:27:47Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1832 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:27:47 dist-test-slave-6bbx krb5kdc[22010](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:27:47 dist-test-slave-6bbx krb5kdc[22010](info): set up 2 sockets
Aug 08 23:27:47 dist-test-slave-6bbx krb5kdc[22010](info): commencing operation
krb5kdc: starting...
W20260808 23:27:48.695228 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:27:48 dist-test-slave-6bbx krb5kdc[22010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:27:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:27:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:27:48.910899 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:35037
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:44719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:35037
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:49.224803 22037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:49.225057 22037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:49.225149 22037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:49.231981 22037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:27:49.232071 22037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:49.232122 22037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:49.232178 22037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:27:49.232220 22037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:27:49.241405 22037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:44719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:35037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:35037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:49.242561 22037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:49.243844 22037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:49.254799 22044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:49.254837 22046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:49.254750 22052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:49.255769 22037 server_base.cc:1034] running on GCE node
I20260808 23:27:49.256542 22037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:49.257818 22037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:49.259027 22037 hybrid_clock.cc:648] HybridClock initialized: now 1786231669258978 us; error 35 us; skew 500 ppm
I20260808 23:27:49.259344 22037 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:49 dist-test-slave-6bbx krb5kdc[22010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231669, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:49.262298 22037 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:27:49.263607 22037 webserver.cc:460] Webserver started at http://127.9.44.126:39815/ using document root <none> and password file <none>
I20260808 23:27:49.264458 22037 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:49.264608 22037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:49.264942 22037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:49.267742 22037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "1a647f287b26429eac924e5f5540e3d2"
format_stamp: "Formatted at 2026-08-08 23:27:49 on dist-test-slave-6bbx"
server_key: "8bf4faa561e157a2ec6f4bcaf98543a8"
server_key_iv: "0e16c647a3de68835bbaf446d6575763"
server_key_version: "encryptionkey@0"
I20260808 23:27:49.268501 22037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "1a647f287b26429eac924e5f5540e3d2"
format_stamp: "Formatted at 2026-08-08 23:27:49 on dist-test-slave-6bbx"
server_key: "8bf4faa561e157a2ec6f4bcaf98543a8"
server_key_iv: "0e16c647a3de68835bbaf446d6575763"
server_key_version: "encryptionkey@0"
I20260808 23:27:49.273201 22037 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 23:27:49.276324 22065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:49.277503 22037 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 23:27:49.277660 22037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "1a647f287b26429eac924e5f5540e3d2"
format_stamp: "Formatted at 2026-08-08 23:27:49 on dist-test-slave-6bbx"
server_key: "8bf4faa561e157a2ec6f4bcaf98543a8"
server_key_iv: "0e16c647a3de68835bbaf446d6575763"
server_key_version: "encryptionkey@0"
I20260808 23:27:49.277838 22037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:49.296042 22037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:49.299966 22037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:49.300212 22037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:49.316149 22037 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:35037
I20260808 23:27:49.316195 22150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:35037 every 8 connection(s)
I20260808 23:27:49.317402 22037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:27:49.319178 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 22037
I20260808 23:27:49.319412 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:27:49.320258 9393 external_mini_cluster.cc:1428] Setting key a1ded08f4bcb7d88c64561e0d3af6982
I20260808 23:27:49.322006 22151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 23:27:49 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:49.334728 22151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2: Bootstrap starting.
I20260808 23:27:49.337816 22151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2: Neither blocks nor log segments found. Creating new log.
I20260808 23:27:49.338910 22151 log.cc:824] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:49.341660 22151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2: No bootstrap required, opened a new log
I20260808 23:27:49.342370 22157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:49.322316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52924 (local address 127.9.44.126:35037)
0808 23:27:49.323048 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:49.323062 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:49.323108 (+ 46us) server_negotiation.cc:407] Connection header received
0808 23:27:49.325101 (+ 1993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:49.325129 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:49.325596 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:49.326023 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:49.327292 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:49.329315 (+ 2023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:49.330475 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:49.331464 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:49.331642 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:49.335508 (+ 3866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:49.335516 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:49.335529 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:49.335561 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:49.338389 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:49.339171 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:49.339178 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:49.339183 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:49.339343 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:49.339756 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:49.339759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:49.339761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:49.340269 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:49.340556 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:49.340983 (+ 427us) server_negotiation.cc:299] Negotiation successful
0808 23:27:49.341351 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":289,"threads_started":1}
I20260808 23:27:49.349371 22151 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a647f287b26429eac924e5f5540e3d2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35037 } }
I20260808 23:27:49.349675 22151 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:27:49.349783 22151 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a647f287b26429eac924e5f5540e3d2, State: Initialized, Role: FOLLOWER
I20260808 23:27:49.350322 22151 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a647f287b26429eac924e5f5540e3d2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35037 } }
I20260808 23:27:49.350529 22151 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:27:49.350649 22151 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:27:49.350801 22151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:27:49.352847 22151 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a647f287b26429eac924e5f5540e3d2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35037 } }
I20260808 23:27:49.353288 22151 leader_election.cc:302] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a647f287b26429eac924e5f5540e3d2; no voters:
I20260808 23:27:49.353961 22151 leader_election.cc:288] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:27:49.354067 22159 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:27:49.354702 22159 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [term 1 LEADER]: Becoming Leader. State: Replica: 1a647f287b26429eac924e5f5540e3d2, State: Running, Role: LEADER
I20260808 23:27:49.355268 22159 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a647f287b26429eac924e5f5540e3d2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35037 } }
I20260808 23:27:49.356055 22151 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:27:49.358007 22161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a647f287b26429eac924e5f5540e3d2. Latest consensus state: current_term: 1 leader_uuid: "1a647f287b26429eac924e5f5540e3d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a647f287b26429eac924e5f5540e3d2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35037 } } }
I20260808 23:27:49.358076 22160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a647f287b26429eac924e5f5540e3d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a647f287b26429eac924e5f5540e3d2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35037 } } }
I20260808 23:27:49.358378 22161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:49.358462 22160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:49.360648 22166 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:27:49.366614 22166 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:27:49.374686 22166 catalog_manager.cc:1347] Generated new cluster ID: feed8d19bef64e718e4801317a1cdaa3
I20260808 23:27:49.374791 22166 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:27:49.382660 22166 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:27:49.384478 22166 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:27:49.396569 22166 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2: Generated new TSK 0
I20260808 23:27:49.397298 22166 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.44.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
I20260808 23:27:49.469039 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:0
--local_ip_for_outbound_sockets=127.9.44.65
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:35037
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:44719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:49.813334 22189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:49.813575 22189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:49.813673 22189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:49.820453 22189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:49.820545 22189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:49.820663 22189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:27:49.829720 22189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:44719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:35037
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:49.830762 22189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:49.832041 22189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:49.844092 22202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:49.844511 22206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:49.844547 22204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:49.845088 22189 server_base.cc:1034] running on GCE node
I20260808 23:27:49.845695 22189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:49.846520 22189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:49.847738 22189 hybrid_clock.cc:648] HybridClock initialized: now 1786231669847683 us; error 41 us; skew 500 ppm
I20260808 23:27:49.848021 22189 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:49 dist-test-slave-6bbx krb5kdc[22010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231669, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:49.851020 22189 init.cc:377] Logged in from keytab as kudu/127.9.44.65@KRBTEST.COM (short username kudu)
I20260808 23:27:49.852385 22189 webserver.cc:460] Webserver started at http://127.9.44.65:35979/ using document root <none> and password file <none>
I20260808 23:27:49.853011 22189 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:49.853132 22189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:49.853425 22189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:49.856206 22189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/data/instance:
uuid: "4a6f331f563b468cba18f903a0863f79"
format_stamp: "Formatted at 2026-08-08 23:27:49 on dist-test-slave-6bbx"
server_key: "4e0a66140638035de41c8602635679c8"
server_key_iv: "d3261c2a62f5e6b99afa03ed5cb70528"
server_key_version: "encryptionkey@0"
I20260808 23:27:49.856962 22189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance:
uuid: "4a6f331f563b468cba18f903a0863f79"
format_stamp: "Formatted at 2026-08-08 23:27:49 on dist-test-slave-6bbx"
server_key: "4e0a66140638035de41c8602635679c8"
server_key_iv: "d3261c2a62f5e6b99afa03ed5cb70528"
server_key_version: "encryptionkey@0"
I20260808 23:27:49.861846 22189 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 23:27:49.865099 22219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:49.866340 22189 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 23:27:49.866463 22189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "4a6f331f563b468cba18f903a0863f79"
format_stamp: "Formatted at 2026-08-08 23:27:49 on dist-test-slave-6bbx"
server_key: "4e0a66140638035de41c8602635679c8"
server_key_iv: "d3261c2a62f5e6b99afa03ed5cb70528"
server_key_version: "encryptionkey@0"
I20260808 23:27:49.866644 22189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:49.882553 22189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:49.885964 22189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:49.886188 22189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:49.886991 22189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:49.888214 22189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:49.888288 22189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:49.888384 22189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:49.888427 22189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:49.917817 22189 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:40607
I20260808 23:27:49.917869 22448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:40607 every 8 connection(s)
I20260808 23:27:49.918995 22189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:27:49.923419 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 22189
I20260808 23:27:49.923621 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance
I20260808 23:27:49.924443 9393 external_mini_cluster.cc:1428] Setting key 64204c3e2c122977ce36ac28497c53e2
Aug 08 23:27:49 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231669, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:49.939358 22454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:49.921264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:49551 (local address 127.9.44.126:35037)
0808 23:27:49.921645 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:49.921650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:49.922533 (+ 883us) server_negotiation.cc:407] Connection header received
0808 23:27:49.924990 (+ 2457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:49.924995 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:49.925071 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:49.925187 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:49.926899 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:49.927597 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:49.929709 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:49.930087 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:49.930175 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:49.933935 (+ 3760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:49.933940 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:49.933943 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:49.933974 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:49.936381 (+ 2407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:49.937132 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:49.937135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:49.937137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:49.937261 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:49.937702 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:49.937707 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:49.937710 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:49.938043 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:49.938179 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:49.938812 (+ 633us) server_negotiation.cc:299] Negotiation successful
0808 23:27:49.939022 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":195,"threads_started":1}
I20260808 23:27:49.940071 22455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:49.921685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.65:49551)
0808 23:27:49.922373 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:49.922420 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:49.924681 (+ 2261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:49.925441 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:49.925452 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:49.925984 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:49.926720 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:49.926748 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:49.927778 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:49.927784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:49.929582 (+ 1798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:49.929597 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:49.930294 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:49.930300 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:49.930557 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:49.931200 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:49.931227 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:49.933718 (+ 2491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:49.936509 (+ 2791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:49.936517 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:49.936530 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:49.937036 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:49.937351 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:49.937356 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:49.937359 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:49.937596 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:49.938156 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:49.938162 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:49.938454 (+ 292us) client_negotiation.cc:769] Sending connection context
0808 23:27:49.938664 (+ 210us) client_negotiation.cc:240] Negotiation successful
0808 23:27:49.939024 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":258,"threads_started":1}
I20260808 23:27:49.941422 22450 heartbeater.cc:347] Connected to a master server at 127.9.44.126:35037
I20260808 23:27:49.941720 22450 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:49.942601 22450 heartbeater.cc:511] Master 127.9.44.126:35037 requested a full tablet report, sending...
I20260808 23:27:49.944789 22087 ts_manager.cc:194] Registered new tserver with Master: 4a6f331f563b468cba18f903a0863f79 (127.9.44.65:40607)
I20260808 23:27:49.946175 22087 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.65@KRBTEST.COM'} at 127.9.44.65:49551
WARNING: no policy specified for kudu/127.9.44.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
I20260808 23:27:49.976945 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:0
--local_ip_for_outbound_sockets=127.9.44.66
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:35037
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:44719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:50.293344 22461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:50.293591 22461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:50.293717 22461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:50.300544 22461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:50.300634 22461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:50.300771 22461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:27:50.310030 22461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:44719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:35037
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:50.311092 22461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:50.312296 22461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:50.325356 22472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:50.325596 22468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:50.325994 22469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:50.326328 22461 server_base.cc:1034] running on GCE node
I20260808 23:27:50.326896 22461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:50.327776 22461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:50.328974 22461 hybrid_clock.cc:648] HybridClock initialized: now 1786231670328919 us; error 44 us; skew 500 ppm
I20260808 23:27:50.329250 22461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:50 dist-test-slave-6bbx krb5kdc[22010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231670, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:50.332263 22461 init.cc:377] Logged in from keytab as kudu/127.9.44.66@KRBTEST.COM (short username kudu)
I20260808 23:27:50.333557 22461 webserver.cc:460] Webserver started at http://127.9.44.66:34543/ using document root <none> and password file <none>
I20260808 23:27:50.334192 22461 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:50.334295 22461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:50.334594 22461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:50.337332 22461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/data/instance:
uuid: "bbc594d4a12e4fb1a9bf2a25b6ca0e00"
format_stamp: "Formatted at 2026-08-08 23:27:50 on dist-test-slave-6bbx"
server_key: "f719322ddb1c3f5faebf6130e219836b"
server_key_iv: "ef2defc29dc5a4b1b844fe6932c6ef6b"
server_key_version: "encryptionkey@0"
I20260808 23:27:50.338074 22461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance:
uuid: "bbc594d4a12e4fb1a9bf2a25b6ca0e00"
format_stamp: "Formatted at 2026-08-08 23:27:50 on dist-test-slave-6bbx"
server_key: "f719322ddb1c3f5faebf6130e219836b"
server_key_iv: "ef2defc29dc5a4b1b844fe6932c6ef6b"
server_key_version: "encryptionkey@0"
I20260808 23:27:50.342849 22461 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 23:27:50.346104 22483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:50.347311 22461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:27:50.347460 22461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "bbc594d4a12e4fb1a9bf2a25b6ca0e00"
format_stamp: "Formatted at 2026-08-08 23:27:50 on dist-test-slave-6bbx"
server_key: "f719322ddb1c3f5faebf6130e219836b"
server_key_iv: "ef2defc29dc5a4b1b844fe6932c6ef6b"
server_key_version: "encryptionkey@0"
I20260808 23:27:50.347625 22461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:50.358295 22461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:50.361750 22461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:50.362007 22461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:50.362874 22461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:50.364153 22461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:50.364249 22461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:50.364372 22461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:50.364429 22461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:50.394560 22461 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:38953
I20260808 23:27:50.394636 22672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:38953 every 8 connection(s)
I20260808 23:27:50.395920 22461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:27:50.404996 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 22461
I20260808 23:27:50.405140 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance
I20260808 23:27:50.405586 9393 external_mini_cluster.cc:1428] Setting key dd331807f136157584954b1ac833a941
Aug 08 23:27:50 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231670, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:50.415932 22454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:50.398489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:54289 (local address 127.9.44.126:35037)
0808 23:27:50.398699 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:50.398704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:50.399910 (+ 1206us) server_negotiation.cc:407] Connection header received
0808 23:27:50.401773 (+ 1863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:50.401778 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:50.401866 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:50.401954 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:50.403495 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:50.404249 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:50.405902 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:50.406345 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:50.406486 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:50.410035 (+ 3549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:50.410041 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:50.410046 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:50.410086 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:50.413036 (+ 2950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:50.413748 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:50.413753 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:50.413756 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:50.413915 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:50.414286 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:50.414290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:50.414293 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:50.414625 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:50.414775 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:50.415365 (+ 590us) server_negotiation.cc:299] Negotiation successful
0808 23:27:50.415582 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 23:27:50.416760 22683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:50.398982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.66:54289)
0808 23:27:50.399746 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:50.399789 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:50.401496 (+ 1707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:50.402228 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:50.402237 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:50.402750 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:50.403313 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:50.403331 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:50.404409 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:50.404413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:50.405749 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:50.405763 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:50.406583 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:50.406590 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:50.406813 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:50.407486 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:50.407510 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:50.409802 (+ 2292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:50.413184 (+ 3382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:50.413191 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:50.413204 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:50.413639 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:50.414006 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:50.414010 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:50.414012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:50.414191 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:50.414731 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:50.414736 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:50.415030 (+ 294us) client_negotiation.cc:769] Sending connection context
0808 23:27:50.415256 (+ 226us) client_negotiation.cc:240] Negotiation successful
0808 23:27:50.415636 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":300,"threads_started":1}
I20260808 23:27:50.418099 22674 heartbeater.cc:347] Connected to a master server at 127.9.44.126:35037
I20260808 23:27:50.418447 22674 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:50.419152 22674 heartbeater.cc:511] Master 127.9.44.126:35037 requested a full tablet report, sending...
I20260808 23:27:50.420635 22087 ts_manager.cc:194] Registered new tserver with Master: bbc594d4a12e4fb1a9bf2a25b6ca0e00 (127.9.44.66:38953)
I20260808 23:27:50.421435 22087 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.66@KRBTEST.COM'} at 127.9.44.66:54289
WARNING: no policy specified for kudu/127.9.44.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
I20260808 23:27:50.457062 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:0
--local_ip_for_outbound_sockets=127.9.44.67
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:35037
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:44719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:50.774792 22687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:50.775063 22687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:50.775205 22687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:50.782150 22687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:50.782259 22687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:50.782403 22687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:27:50.791571 22687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:44719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:35037
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:50.792702 22687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:50.793936 22687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:50.805908 22701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:50.806046 22698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:50.806398 22697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:50.806986 22687 server_base.cc:1034] running on GCE node
I20260808 23:27:50.807540 22687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:50.808408 22687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:50.809590 22687 hybrid_clock.cc:648] HybridClock initialized: now 1786231670809535 us; error 45 us; skew 500 ppm
I20260808 23:27:50.809849 22687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:50 dist-test-slave-6bbx krb5kdc[22010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231670, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:50.812769 22687 init.cc:377] Logged in from keytab as kudu/127.9.44.67@KRBTEST.COM (short username kudu)
I20260808 23:27:50.813997 22687 webserver.cc:460] Webserver started at http://127.9.44.67:44287/ using document root <none> and password file <none>
I20260808 23:27:50.814602 22687 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:50.814723 22687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:50.815016 22687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:27:50.817833 22687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/data/instance:
uuid: "e07d246604554a03ab44f2b9c4b8b0b9"
format_stamp: "Formatted at 2026-08-08 23:27:50 on dist-test-slave-6bbx"
server_key: "d60ec96b2d63ba924f738d9508e48909"
server_key_iv: "43e4954fcaf1d6c28ece3c80131f78de"
server_key_version: "encryptionkey@0"
I20260808 23:27:50.818576 22687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance:
uuid: "e07d246604554a03ab44f2b9c4b8b0b9"
format_stamp: "Formatted at 2026-08-08 23:27:50 on dist-test-slave-6bbx"
server_key: "d60ec96b2d63ba924f738d9508e48909"
server_key_iv: "43e4954fcaf1d6c28ece3c80131f78de"
server_key_version: "encryptionkey@0"
I20260808 23:27:50.823395 22687 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 23:27:50.826735 22718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:50.827993 22687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:27:50.828137 22687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "e07d246604554a03ab44f2b9c4b8b0b9"
format_stamp: "Formatted at 2026-08-08 23:27:50 on dist-test-slave-6bbx"
server_key: "d60ec96b2d63ba924f738d9508e48909"
server_key_iv: "43e4954fcaf1d6c28ece3c80131f78de"
server_key_version: "encryptionkey@0"
I20260808 23:27:50.828322 22687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:50.841118 22687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:50.844980 22687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:50.845238 22687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:50.846202 22687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:50.847446 22687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:50.847524 22687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:50.847654 22687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:50.847785 22687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260808 23:27:50.878255 22687 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:41061
I20260808 23:27:50.878298 22920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:41061 every 8 connection(s)
I20260808 23:27:50.879487 22687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:27:50.886981 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 22687
I20260808 23:27:50.887138 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance
I20260808 23:27:50.887574 9393 external_mini_cluster.cc:1428] Setting key fc24e341074990b86559a7bf22cea323
Aug 08 23:27:50 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231670, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:27:50.898175 22454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:50.881964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:38881 (local address 127.9.44.126:35037)
0808 23:27:50.882142 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:50.882146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:50.883307 (+ 1161us) server_negotiation.cc:407] Connection header received
0808 23:27:50.885106 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:50.885111 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:50.885192 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:50.885282 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:50.886723 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:50.887535 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:50.889389 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:50.889920 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:50.890018 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:50.893372 (+ 3354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:50.893377 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:50.893380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:50.893412 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:50.895733 (+ 2321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:50.896357 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:50.896361 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:50.896363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:50.896485 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:50.896807 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:50.896810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:50.896812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:50.897070 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:50.897181 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:50.897759 (+ 578us) server_negotiation.cc:299] Negotiation successful
0808 23:27:50.897935 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 23:27:50.899081 22928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:50.882468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.67:38881)
0808 23:27:50.883154 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:50.883190 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:50.884842 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:50.885509 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:50.885517 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:50.886002 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:50.886548 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:50.886564 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:50.887720 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:50.887725 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:50.889265 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:50.889277 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:50.890141 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:50.890149 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:50.890453 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:50.891116 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:50.891139 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:50.893175 (+ 2036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:27:50.895851 (+ 2676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:50.895858 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:50.895868 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:50.896271 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:50.896557 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:27:50.896560 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:27:50.896562 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:27:50.896729 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:27:50.897158 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:27:50.897163 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:27:50.897444 (+ 281us) client_negotiation.cc:769] Sending connection context
0808 23:27:50.897642 (+ 198us) client_negotiation.cc:240] Negotiation successful
0808 23:27:50.897980 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":270,"threads_started":1}
I20260808 23:27:50.900460 22923 heartbeater.cc:347] Connected to a master server at 127.9.44.126:35037
I20260808 23:27:50.900745 22923 heartbeater.cc:464] Registering TS with master...
I20260808 23:27:50.901449 22923 heartbeater.cc:511] Master 127.9.44.126:35037 requested a full tablet report, sending...
I20260808 23:27:50.902910 22087 ts_manager.cc:194] Registered new tserver with Master: e07d246604554a03ab44f2b9c4b8b0b9 (127.9.44.67:41061)
I20260808 23:27:50.903692 22087 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.67@KRBTEST.COM'} at 127.9.44.67:38881
I20260808 23:27:50.914700 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.44.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.44.65@KRBTEST.COM" created.
I20260808 23:27:50.949378 22450 heartbeater.cc:503] Master 127.9.44.126:35037 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.44.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.65.keytab.
Entry for principal oryx/127.9.44.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.65.keytab.
WARNING: no policy specified for oryx/127.9.44.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.44.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.44.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.66.keytab.
Entry for principal oryx/127.9.44.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.66.keytab.
WARNING: no policy specified for oryx/127.9.44.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.44.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.44.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.67.keytab.
Entry for principal oryx/127.9.44.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.67.keytab.
I20260808 23:27:51.047972 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 22189
Aug 08 23:27:51 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.65@KRBTEST.COM
I20260808 23:27:51.063546 22941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:51.049126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55580 (local address 127.9.44.65:40607)
0808 23:27:51.049873 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:51.049881 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:51.049908 (+ 27us) server_negotiation.cc:407] Connection header received
0808 23:27:51.049974 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:51.049981 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:51.050200 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:51.050328 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:51.051705 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:51.052829 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:51.054114 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:51.054666 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:51.054760 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:51.058653 (+ 3893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:51.058665 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:51.058679 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:51.058712 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:51.061266 (+ 2554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:51.061894 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:51.061900 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:51.061902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:51.062023 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:51.062327 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:51.062330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:51.062332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:51.062648 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:51.062884 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:51.063074 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 23:27:51.063265 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":402,"threads_started":1}
I20260808 23:27:51.064800 22440 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:51.151626 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 22189
I20260808 23:27:51.175156 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 22461
Aug 08 23:27:51 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.66@KRBTEST.COM
I20260808 23:27:51.190313 22945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:51.176036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39396 (local address 127.9.44.66:38953)
0808 23:27:51.176542 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:51.176549 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:51.176575 (+ 26us) server_negotiation.cc:407] Connection header received
0808 23:27:51.176642 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:51.176649 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:51.176856 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:51.176982 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:51.178164 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:51.179159 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:51.180424 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:51.180973 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:51.181067 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:51.184965 (+ 3898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:51.184976 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:51.184990 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:51.185029 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:51.187592 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:51.188239 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:51.188246 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:51.188248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:51.188386 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:51.188746 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:51.188750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:51.188752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:51.189082 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:51.189388 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:51.189717 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 23:27:51.189973 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":251,"threads_started":1}
I20260808 23:27:51.191489 22668 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:51.271523 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 22461
I20260808 23:27:51.298127 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 22687
Aug 08 23:27:51 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.67@KRBTEST.COM
I20260808 23:27:51.321205 22947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:51.299532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44374 (local address 127.9.44.67:41061)
0808 23:27:51.300282 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:51.300289 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:51.300319 (+ 30us) server_negotiation.cc:407] Connection header received
0808 23:27:51.300387 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:51.300396 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:51.300641 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:51.300821 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:51.302511 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:51.303694 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:51.305470 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:51.306407 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:51.306548 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:51.310661 (+ 4113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:51.310678 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:51.310699 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:51.310743 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:51.314474 (+ 3731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:51.315155 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:51.315165 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:51.315169 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:51.315335 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:51.319751 (+ 4416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:51.319756 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:51.319758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:51.320103 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:51.320365 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:51.320674 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 23:27:51.320878 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":417,"threads_started":1}
I20260808 23:27:51.322518 22911 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:51.402549 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 22687
I20260808 23:27:51.422608 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 22037
I20260808 23:27:51.423800 22139 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:27:51.523460 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 22037
I20260808 23:27:51.543586 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:35037
--webserver_interface=127.9.44.126
--webserver_port=39815
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:44719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:35037
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:27:51.861172 22951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:51.861430 22951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:51.861523 22951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:51.868324 22951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:27:51.868415 22951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:51.868466 22951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:51.868520 22951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:27:51.868564 22951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:27:51.877693 22951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:44719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:35037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:35037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=39815
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:51.878811 22951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:51.880015 22951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:51.891121 22963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:51.891172 22961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:51.891203 22965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:51.891930 22951 server_base.cc:1034] running on GCE node
I20260808 23:27:51.892647 22951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:51.893905 22951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:51.895107 22951 hybrid_clock.cc:648] HybridClock initialized: now 1786231671895049 us; error 44 us; skew 500 ppm
I20260808 23:27:51.895426 22951 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:51 dist-test-slave-6bbx krb5kdc[22010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231671, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:51.898274 22951 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:27:51.899643 22951 webserver.cc:460] Webserver started at http://127.9.44.126:39815/ using document root <none> and password file <none>
I20260808 23:27:51.900305 22951 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:51.900444 22951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:51.905468 22951 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 23:27:51.908049 22981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:51.909236 22951 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260808 23:27:51.909401 22951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "1a647f287b26429eac924e5f5540e3d2"
format_stamp: "Formatted at 2026-08-08 23:27:49 on dist-test-slave-6bbx"
server_key: "8bf4faa561e157a2ec6f4bcaf98543a8"
server_key_iv: "0e16c647a3de68835bbaf446d6575763"
server_key_version: "encryptionkey@0"
I20260808 23:27:51.910279 22951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:51.936806 22951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:51.940320 22951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:51.940562 22951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:51.957043 22951 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:35037
I20260808 23:27:51.957073 23071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:35037 every 8 connection(s)
I20260808 23:27:51.958410 22951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:27:51.964789 23072 sys_catalog.cc:263] Verifying existing consensus state
I20260808 23:27:51.965901 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 22951
I20260808 23:27:51.966881 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:40607
--local_ip_for_outbound_sockets=127.9.44.65
--tserver_master_addrs=127.9.44.126:35037
--webserver_port=35979
--webserver_interface=127.9.44.65
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:44719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.44.65
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:27:51.968931 23072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2: Bootstrap starting.
I20260808 23:27:51.988744 23072 log.cc:824] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2: Log is configured to *not* fsync() on all Append() calls
I20260808 23:27:51.997184 23072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:27:51.997671 23072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2: Bootstrap complete.
I20260808 23:27:52.006309 23072 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a647f287b26429eac924e5f5540e3d2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35037 } }
I20260808 23:27:52.007067 23072 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a647f287b26429eac924e5f5540e3d2, State: Initialized, Role: FOLLOWER
I20260808 23:27:52.007652 23072 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a647f287b26429eac924e5f5540e3d2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35037 } }
I20260808 23:27:52.007905 23072 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:27:52.008006 23072 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:27:52.008170 23072 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [term 1 FOLLOWER]: Advancing to term 2
I20260808 23:27:52.011600 23072 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a647f287b26429eac924e5f5540e3d2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35037 } }
I20260808 23:27:52.012111 23072 leader_election.cc:302] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a647f287b26429eac924e5f5540e3d2; no voters:
I20260808 23:27:52.012671 23072 leader_election.cc:288] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 23:27:52.012799 23079 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 23:27:52.013846 23079 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [term 2 LEADER]: Becoming Leader. State: Replica: 1a647f287b26429eac924e5f5540e3d2, State: Running, Role: LEADER
I20260808 23:27:52.014397 23079 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a647f287b26429eac924e5f5540e3d2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35037 } }
I20260808 23:27:52.014819 23072 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:27:52.016950 23081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a647f287b26429eac924e5f5540e3d2. Latest consensus state: current_term: 2 leader_uuid: "1a647f287b26429eac924e5f5540e3d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a647f287b26429eac924e5f5540e3d2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35037 } } }
I20260808 23:27:52.017011 23080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1a647f287b26429eac924e5f5540e3d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a647f287b26429eac924e5f5540e3d2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35037 } } }
I20260808 23:27:52.017390 23080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:52.017295 23081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2 [sys.catalog]: This master's current role is: LEADER
I20260808 23:27:52.029675 23086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:27:52.035079 23086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:27:52.036566 23086 catalog_manager.cc:1259] Loaded cluster ID: feed8d19bef64e718e4801317a1cdaa3
I20260808 23:27:52.036761 23086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:27:52.040915 23086 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:27:52.042392 23086 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1a647f287b26429eac924e5f5540e3d2: Loaded TSK: 0
I20260808 23:27:52.043937 23086 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 23:27:52.304786 23076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:52.305056 23076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:52.305187 23076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:52.312132 23076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:52.312237 23076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:52.312389 23076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:27:52.321779 23076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:44719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.65.keytab
--principal=oryx/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:40607
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=35979
--tserver_master_addrs=127.9.44.126:35037
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:52.322992 23076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:52.324250 23076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:52.337782 23118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:52.337776 23113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:52.338783 23125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:52.339025 23076 server_base.cc:1034] running on GCE node
I20260808 23:27:52.339603 23076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:52.340488 23076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:52.341688 23076 hybrid_clock.cc:648] HybridClock initialized: now 1786231672341630 us; error 54 us; skew 500 ppm
I20260808 23:27:52.341984 23076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:52 dist-test-slave-6bbx krb5kdc[22010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231672, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:52.345131 23076 init.cc:377] Logged in from keytab as oryx/127.9.44.65@KRBTEST.COM (short username oryx)
I20260808 23:27:52.346483 23076 webserver.cc:460] Webserver started at http://127.9.44.65:35979/ using document root <none> and password file <none>
I20260808 23:27:52.347110 23076 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:52.347242 23076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:52.352555 23076 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:27:52.355152 23134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:52.356459 23076 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 23:27:52.356606 23076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "4a6f331f563b468cba18f903a0863f79"
format_stamp: "Formatted at 2026-08-08 23:27:49 on dist-test-slave-6bbx"
server_key: "4e0a66140638035de41c8602635679c8"
server_key_iv: "d3261c2a62f5e6b99afa03ed5cb70528"
server_key_version: "encryptionkey@0"
I20260808 23:27:52.357503 23076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:52.382829 23076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:52.386344 23076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:52.386615 23076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:52.387513 23076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:52.388782 23076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:52.388882 23076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:52.388988 23076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:52.389057 23076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:52.419308 23076 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:40607
I20260808 23:27:52.419380 23341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:40607 every 8 connection(s)
I20260808 23:27:52.420747 23076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:27:52.423835 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 23076
I20260808 23:27:52.424552 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:38953
--local_ip_for_outbound_sockets=127.9.44.66
--tserver_master_addrs=127.9.44.126:35037
--webserver_port=34543
--webserver_interface=127.9.44.66
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:44719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.44.66
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:27:52 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:52 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 23:27:52.439738 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:52.423697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:56385 (local address 127.9.44.126:35037)
0808 23:27:52.424453 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:52.424468 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:52.425004 (+ 536us) server_negotiation.cc:407] Connection header received
0808 23:27:52.427121 (+ 2117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:52.427140 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:52.427549 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:52.427951 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:52.429299 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:52.431028 (+ 1729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:52.432576 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:52.433459 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:52.433632 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:52.438905 (+ 5273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.65:56385: BlockingRecv error: recv got EOF from 127.9.44.65:56385 (error 108)
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":260,"threads_started":1}
I20260808 23:27:52.439770 23353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:52.423755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.65:56385)
0808 23:27:52.424843 (+ 1088us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:52.424890 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:52.426555 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:52.428058 (+ 1503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:52.428065 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:52.428567 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:52.429100 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:52.429117 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:52.431163 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:52.431167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:52.432438 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:52.432451 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:52.433637 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:52.433642 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:52.433829 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:52.434439 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:52.434461 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:52.438622 (+ 4161us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":811,"thread_start_us":620,"threads_started":1}
W20260808 23:27:52.440423 23346 heartbeater.cc:644] Failed to heartbeat to 127.9.44.126:35037 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.44.126:35037: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Aug 08 23:27:52 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:52 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:27:52.448720 23353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:52.440941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.65:32835)
0808 23:27:52.441115 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:52.441134 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:52.441228 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:52.441574 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:52.441577 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:52.441864 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:52.442001 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:52.442008 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:52.443308 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:52.443311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:52.444208 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:52.444220 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:52.445059 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:52.445063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:52.445176 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:52.445679 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:52.445695 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:52.448333 (+ 2638us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 23:27:52.448791 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:52.441153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:32835 (local address 127.9.44.126:35037)
0808 23:27:52.441320 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:52.441326 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:52.441343 (+ 17us) server_negotiation.cc:407] Connection header received
0808 23:27:52.441399 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:52.441403 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:52.441466 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:52.441567 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:52.442146 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:52.443193 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:52.444349 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:52.444944 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:52.445060 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:52.448580 (+ 3520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.65:32835: BlockingRecv error: recv got EOF from 127.9.44.65:32835 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 23:27:52 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:52 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:27:52.459743 23353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:52.449442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.65:43499)
0808 23:27:52.449655 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:52.449672 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:52.449768 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:52.450087 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:52.450090 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:52.450344 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:52.450473 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:52.450480 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:52.451792 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:52.451796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:52.452651 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:52.452662 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:52.453522 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:52.453526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:52.453635 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:52.456973 (+ 3338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:52.456988 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:52.459495 (+ 2507us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116}
W20260808 23:27:52.459997 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:52.449655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:43499 (local address 127.9.44.126:35037)
0808 23:27:52.449820 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:52.449825 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:52.449843 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:27:52.449893 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:52.449898 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:52.449978 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:52.450080 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:52.450595 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:52.451638 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:52.452792 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:52.453404 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:52.453537 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:52.459756 (+ 6219us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.65:43499: BlockingRecv error: recv got EOF from 127.9.44.65:43499 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260808 23:27:52.460119 23346 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 23:27:52.758762 23354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:52.759014 23354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:52.759114 23354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:52.766098 23354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:52.766197 23354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:52.766309 23354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:27:52.775381 23354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:44719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.66.keytab
--principal=oryx/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:38953
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=34543
--tserver_master_addrs=127.9.44.126:35037
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:52.776515 23354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:52.777699 23354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:52.789731 23363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:52.789687 23367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:52.789763 23371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:52.790652 23354 server_base.cc:1034] running on GCE node
I20260808 23:27:52.791213 23354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:52.792119 23354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:52.793300 23354 hybrid_clock.cc:648] HybridClock initialized: now 1786231672793245 us; error 48 us; skew 500 ppm
I20260808 23:27:52.793552 23354 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:52 dist-test-slave-6bbx krb5kdc[22010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231672, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:52.796455 23354 init.cc:377] Logged in from keytab as oryx/127.9.44.66@KRBTEST.COM (short username oryx)
I20260808 23:27:52.797720 23354 webserver.cc:460] Webserver started at http://127.9.44.66:34543/ using document root <none> and password file <none>
I20260808 23:27:52.798357 23354 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:52.798481 23354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:52.803484 23354 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:27:52.805956 23386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:52.807183 23354 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 23:27:52.807348 23354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "bbc594d4a12e4fb1a9bf2a25b6ca0e00"
format_stamp: "Formatted at 2026-08-08 23:27:50 on dist-test-slave-6bbx"
server_key: "f719322ddb1c3f5faebf6130e219836b"
server_key_iv: "ef2defc29dc5a4b1b844fe6932c6ef6b"
server_key_version: "encryptionkey@0"
I20260808 23:27:52.808295 23354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:52.829679 23354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:52.833124 23354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:52.833359 23354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:52.834225 23354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:52.835443 23354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:52.835539 23354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:52.835641 23354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:52.835747 23354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260808 23:27:52.865288 23354 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:38953
I20260808 23:27:52.865350 23586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:38953 every 8 connection(s)
I20260808 23:27:52.866660 23354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:27:52.872746 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 23354
I20260808 23:27:52.873479 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:41061
--local_ip_for_outbound_sockets=127.9.44.67
--tserver_master_addrs=127.9.44.126:35037
--webserver_port=44287
--webserver_interface=127.9.44.67
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:44719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.44.67
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:27:52 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:52 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 23:27:52.882938 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:52.869076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:38419 (local address 127.9.44.126:35037)
0808 23:27:52.869274 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:52.869280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:52.870398 (+ 1118us) server_negotiation.cc:407] Connection header received
0808 23:27:52.872221 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:52.872225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:52.872293 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:52.872383 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:52.874365 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:52.875082 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:52.877085 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:52.877481 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:52.877576 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:52.882738 (+ 5162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.66:38419: BlockingRecv error: recv got EOF from 127.9.44.66:38419 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 23:27:52.883785 23589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:52.869537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.66:38419)
0808 23:27:52.870235 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:52.870281 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:52.871986 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:52.872665 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:52.872676 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:52.873381 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:52.874173 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:52.874199 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:52.875258 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:52.875263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:52.876959 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:52.876974 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:52.877687 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:52.877693 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:52.877948 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:52.878593 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:52.878621 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:52.882554 (+ 3933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":278,"threads_started":1}
W20260808 23:27:52.884460 23587 heartbeater.cc:644] Failed to heartbeat to 127.9.44.126:35037 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.44.126:35037: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Aug 08 23:27:52 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:52 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:27:52.892788 23589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:52.885009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.66:49379)
0808 23:27:52.885173 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:52.885191 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:52.885284 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:52.885531 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:52.885535 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:52.885932 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:52.886099 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:52.886110 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:52.887038 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:52.887044 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:52.888343 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:52.888358 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:52.889037 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:52.889043 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:52.889198 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:52.889735 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:52.889755 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:52.892523 (+ 2768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 23:27:52.893110 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:52.885143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:49379 (local address 127.9.44.126:35037)
0808 23:27:52.885297 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:52.885301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:52.885317 (+ 16us) server_negotiation.cc:407] Connection header received
0808 23:27:52.885360 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:52.885364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:52.885417 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:52.885492 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:52.886224 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:52.886893 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:52.888479 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:52.888884 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:52.888977 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:52.892746 (+ 3769us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.66:49379: BlockingRecv error: recv got EOF from 127.9.44.66:49379 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 23:27:52 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:52 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:27:52.901310 23589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:52.893432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.66:46707)
0808 23:27:52.893576 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:52.893591 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:52.893692 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:52.894134 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:52.894139 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:52.894471 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:52.894673 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:52.894686 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:52.895578 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:52.895583 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:52.896984 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:52.897000 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:52.897653 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:52.897659 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:52.897829 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:52.898346 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:52.898368 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:52.901119 (+ 2751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 23:27:52.901605 23587 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 23:27:52.901973 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:52.893752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:46707 (local address 127.9.44.126:35037)
0808 23:27:52.893892 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:52.893896 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:52.893909 (+ 13us) server_negotiation.cc:407] Connection header received
0808 23:27:52.893945 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:52.893948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:52.893998 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:52.894070 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:52.894794 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:52.895442 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:52.897122 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:52.897516 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:52.897605 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:52.901365 (+ 3760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.66:46707: BlockingRecv error: recv got EOF from 127.9.44.66:46707 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260808 23:27:53.208745 23592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:27:53.208990 23592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:27:53.209088 23592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:27:53.216048 23592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:27:53.216140 23592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:27:53.216250 23592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:27:53.225281 23592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:44719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/krb5kdc/oryx_127.9.44.67.keytab
--principal=oryx/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:41061
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=44287
--tserver_master_addrs=127.9.44.126:35037
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:27:53.226325 23592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:27:53.227494 23592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:27:53.239557 23607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:53.239635 23603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:27:53.239692 23602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:27:53.241223 23592 server_base.cc:1034] running on GCE node
I20260808 23:27:53.241770 23592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:27:53.242611 23592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:27:53.243755 23592 hybrid_clock.cc:648] HybridClock initialized: now 1786231673243716 us; error 27 us; skew 500 ppm
I20260808 23:27:53.244037 23592 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:27:53 dist-test-slave-6bbx krb5kdc[22010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231673, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:27:53.246924 23592 init.cc:377] Logged in from keytab as oryx/127.9.44.67@KRBTEST.COM (short username oryx)
I20260808 23:27:53.248183 23592 webserver.cc:460] Webserver started at http://127.9.44.67:44287/ using document root <none> and password file <none>
I20260808 23:27:53.248785 23592 fs_manager.cc:359] Metadata directory not provided
I20260808 23:27:53.248891 23592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:27:53.253821 23592 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 23:27:53.256249 23616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:53.257437 23592 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 23:27:53.257583 23592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "e07d246604554a03ab44f2b9c4b8b0b9"
format_stamp: "Formatted at 2026-08-08 23:27:50 on dist-test-slave-6bbx"
server_key: "d60ec96b2d63ba924f738d9508e48909"
server_key_iv: "43e4954fcaf1d6c28ece3c80131f78de"
server_key_version: "encryptionkey@0"
I20260808 23:27:53.258489 23592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:27:53.274415 23592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:27:53.277813 23592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:27:53.278048 23592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:27:53.278892 23592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:27:53.280125 23592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:27:53.280218 23592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:53.280321 23592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:27:53.280401 23592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:27:53.309913 23592 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:41061
I20260808 23:27:53.309985 23827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:41061 every 8 connection(s)
I20260808 23:27:53.311363 23592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:27:53.312013 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 23592
Aug 08 23:27:53 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:53 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 23:27:53.328997 23834 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:53.314457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:46179 (local address 127.9.44.126:35037)
0808 23:27:53.315012 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:53.315018 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:53.316114 (+ 1096us) server_negotiation.cc:407] Connection header received
0808 23:27:53.318323 (+ 2209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:53.318326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:53.318378 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:53.318447 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:53.320181 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:53.320904 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:53.322785 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:53.323260 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:53.323358 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:53.328768 (+ 5410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.67:46179: BlockingRecv error: recv got EOF from 127.9.44.67:46179 (error 108)
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":434,"thread_start_us":297,"threads_started":1}
I20260808 23:27:53.329025 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:53.313054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52938 (local address 127.9.44.126:35037)
0808 23:27:53.313243 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:53.313250 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:53.313802 (+ 552us) server_negotiation.cc:407] Connection header received
0808 23:27:53.313991 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:53.313996 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:53.314087 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:53.314219 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:53.316308 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:53.317000 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:53.318742 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:53.319133 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:53.319218 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:53.322016 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:27:53.322023 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:27:53.322037 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:27:53.322077 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:27:53.325660 (+ 3583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:53.326346 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:53.326364 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:53.326373 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:53.326594 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:27:53.327021 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:27:53.327026 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:27:53.327029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:27:53.327620 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:27:53.327979 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0808 23:27:53.328268 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 23:27:53.328627 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 23:27:53.329715 23835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:53.315055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.67:46179)
0808 23:27:53.315964 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:53.316022 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:53.318073 (+ 2051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:53.318687 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:53.318697 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:53.319232 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:53.319998 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:53.320027 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:53.321067 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:53.321073 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:53.322654 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:53.322667 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:53.323482 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:53.323491 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:53.323774 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:53.324308 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:53.324335 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:53.328554 (+ 4219us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":294,"threads_started":1}
W20260808 23:27:53.330287 23830 heartbeater.cc:644] Failed to heartbeat to 127.9.44.126:35037 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.44.126:35037: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Aug 08 23:27:53 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:53 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:27:53.337527 23835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:53.330800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.67:37491)
0808 23:27:53.330961 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:53.330975 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:53.331047 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:53.331358 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:53.331363 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:53.331776 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:53.331917 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:53.331925 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:53.332885 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:53.332891 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:53.333741 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:53.333752 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:53.334395 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:53.334400 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:53.334561 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:53.335009 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:53.335024 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:53.337346 (+ 2322us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 23:27:53.337837 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:53.330870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:37491 (local address 127.9.44.126:35037)
0808 23:27:53.331034 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:53.331039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:53.331056 (+ 17us) server_negotiation.cc:407] Connection header received
0808 23:27:53.331129 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:53.331134 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:53.331226 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:53.331305 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:53.332040 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:53.332762 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:53.333866 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:53.334268 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:53.334369 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:53.337541 (+ 3172us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.67:37491: BlockingRecv error: recv got EOF from 127.9.44.67:37491 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 23:27:53 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:53 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:27:53.344580 23835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:53.338093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.67:38581)
0808 23:27:53.338235 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:53.338250 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:53.338338 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:53.338559 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:53.338562 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:53.338805 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:53.338945 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:53.338952 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:53.339829 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:53.339833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:53.340681 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:53.340691 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:53.341265 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:53.341268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:53.341401 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:53.341804 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:53.341818 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:53.344314 (+ 2496us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 23:27:53.344671 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:53.338128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:38581 (local address 127.9.44.126:35037)
0808 23:27:53.338304 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:53.338308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:53.338323 (+ 15us) server_negotiation.cc:407] Connection header received
0808 23:27:53.338419 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:53.338422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:53.338466 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:53.338533 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:53.339046 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:53.339730 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:53.340784 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:53.341166 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:53.341245 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:53.344509 (+ 3264us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.67:38581: BlockingRecv error: recv got EOF from 127.9.44.67:38581 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
W20260808 23:27:53.344825 23830 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 23:27:53 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:53 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:27:53.468381 23838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:53.460570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.65:40795)
0808 23:27:53.461075 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:53.461094 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:53.461201 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:53.461454 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:53.461457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:53.461954 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:53.462225 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:53.462239 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:53.463173 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:53.463177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:53.464187 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:53.464201 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:53.464887 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:53.464892 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:53.465013 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:53.465509 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:53.465525 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:53.468170 (+ 2645us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":227,"threads_started":1}
W20260808 23:27:53.468569 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:53.460624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:40795 (local address 127.9.44.126:35037)
0808 23:27:53.460788 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:53.460792 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:53.461156 (+ 364us) server_negotiation.cc:407] Connection header received
0808 23:27:53.461284 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:53.461287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:53.461348 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:53.461425 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:53.462360 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:53.463050 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:53.464316 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:53.464747 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:53.464842 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:53.468356 (+ 3514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.65:40795: BlockingRecv error: recv got EOF from 127.9.44.65:40795 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 23:27:53 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:53 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:27:53.910292 23839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:53.902179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.66:33349)
0808 23:27:53.902639 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:53.902655 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:53.902773 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:53.903078 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:53.903081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:53.903739 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:53.903953 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:53.903964 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:53.904983 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:53.904987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:53.905940 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:53.905951 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:53.906629 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:53.906633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:53.906773 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:53.907351 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:53.907369 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:53.910055 (+ 2686us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":234,"threads_started":1}
W20260808 23:27:53.910491 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:53.902246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:33349 (local address 127.9.44.126:35037)
0808 23:27:53.902489 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:53.902495 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:53.902731 (+ 236us) server_negotiation.cc:407] Connection header received
0808 23:27:53.902869 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:53.902873 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:53.902954 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:53.903036 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:53.904101 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:53.904846 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:53.906062 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:53.906493 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:53.906586 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:53.910268 (+ 3682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.66:33349: BlockingRecv error: recv got EOF from 127.9.44.66:33349 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 08 23:27:54 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:54 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 23:27:54.355286 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:54.345484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:35281 (local address 127.9.44.126:35037)
0808 23:27:54.345668 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:54.345674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:54.345995 (+ 321us) server_negotiation.cc:407] Connection header received
0808 23:27:54.346154 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:54.346160 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:54.346249 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:54.346364 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:54.347135 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:54.348372 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:54.349629 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:54.350303 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:54.350441 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:54.355105 (+ 4664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.67:35281: BlockingRecv error: recv got EOF from 127.9.44.67:35281 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 23:27:54.355307 23840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:54.345393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.67:35281)
0808 23:27:54.345909 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:54.345925 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:54.346037 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:54.346377 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:54.346380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:54.346826 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:54.347003 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:54.347011 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:54.348517 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:54.348521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:54.349493 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:54.349505 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:54.350470 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:54.350474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:54.350583 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:54.351538 (+ 955us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:54.351557 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:54.354950 (+ 3393us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":261,"threads_started":1}
Aug 08 23:27:54 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:54 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:27:54.477120 23842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:54.469193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.65:38137)
0808 23:27:54.469654 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:54.469669 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:54.469795 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:54.470121 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:54.470125 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:54.470573 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:54.470724 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:54.470735 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:54.471799 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:54.471803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:54.472710 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:54.472722 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:54.473441 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:54.473447 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:54.473604 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:54.474103 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:54.474121 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:54.476807 (+ 2686us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":246,"threads_started":1}
W20260808 23:27:54.477258 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:54.469223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:38137 (local address 127.9.44.126:35037)
0808 23:27:54.469420 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:54.469426 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:54.469750 (+ 324us) server_negotiation.cc:407] Connection header received
0808 23:27:54.469913 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:54.469919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:54.469989 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:54.470070 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:54.470877 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:54.471616 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:54.472851 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:54.473287 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:54.473396 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:54.477026 (+ 3630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.65:38137: BlockingRecv error: recv got EOF from 127.9.44.65:38137 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 23:27:54 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:54 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:27:54.919281 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:54.911155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.66:33209)
0808 23:27:54.911689 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:54.911711 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:54.911831 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:54.912140 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:54.912144 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:54.912612 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:54.912796 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:54.912807 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:54.913845 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:54.913850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:54.914844 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:54.914858 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:54.915545 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:54.915550 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:54.915700 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:54.916292 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:54.916312 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:54.919027 (+ 2715us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":278,"threads_started":1}
W20260808 23:27:54.919423 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:54.911219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:33209 (local address 127.9.44.126:35037)
0808 23:27:54.911416 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:54.911423 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:54.911772 (+ 349us) server_negotiation.cc:407] Connection header received
0808 23:27:54.911921 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:54.911925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:54.912001 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:54.912082 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:54.912915 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:54.913713 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:54.914956 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:54.915409 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:54.915496 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:54.919211 (+ 3715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.66:33209: BlockingRecv error: recv got EOF from 127.9.44.66:33209 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 23:27:55 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:55 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:27:55.363777 23846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:55.356164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.67:32957)
0808 23:27:55.356599 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:55.356614 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:55.356729 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:55.356990 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:55.356993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:55.357394 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:55.357542 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:55.357551 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:55.358475 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:55.358478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:55.359367 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:55.359379 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:55.360027 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:55.360031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:55.360139 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:55.360714 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:55.360733 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:55.363504 (+ 2771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":230,"threads_started":1}
W20260808 23:27:55.363906 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:55.356303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:32957 (local address 127.9.44.126:35037)
0808 23:27:55.356518 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:55.356525 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:55.356683 (+ 158us) server_negotiation.cc:407] Connection header received
0808 23:27:55.356811 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:55.356814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:55.356884 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:55.356960 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:55.357656 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:55.358360 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:55.359476 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:55.359920 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:55.360011 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:55.363691 (+ 3680us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.67:32957: BlockingRecv error: recv got EOF from 127.9.44.67:32957 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 08 23:27:55 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:55 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:27:55.486088 23847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:55.477954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.65:34479)
0808 23:27:55.478384 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:55.478399 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:55.478517 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:55.478775 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:55.478779 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:55.479147 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:55.479331 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:55.479341 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:55.480288 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:55.480292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:55.481411 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:55.481425 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:55.482249 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:55.482257 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:55.482503 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:55.483099 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:55.483117 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:55.485767 (+ 2650us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":220,"threads_started":1}
W20260808 23:27:55.486209 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:55.478018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:34479 (local address 127.9.44.126:35037)
0808 23:27:55.478224 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:55.478231 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:55.478474 (+ 243us) server_negotiation.cc:407] Connection header received
0808 23:27:55.478603 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:55.478606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:55.478668 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:55.478742 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:55.479441 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:55.480191 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:55.481550 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:55.482074 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:55.482194 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:55.485977 (+ 3783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.65:34479: BlockingRecv error: recv got EOF from 127.9.44.65:34479 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 23:27:55 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:55 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:27:55.927819 23850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:55.920218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.66:50843)
0808 23:27:55.920693 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:55.920711 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:55.920839 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:55.921150 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:55.921154 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:55.921523 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:55.921702 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:55.921712 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:55.922693 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:55.922696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:55.923595 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:55.923607 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:55.924326 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:55.924331 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:55.924464 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:55.925014 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:55.925031 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:55.927539 (+ 2508us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":255,"threads_started":1}
W20260808 23:27:55.927954 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:55.920317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:50843 (local address 127.9.44.126:35037)
0808 23:27:55.920522 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:55.920529 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:55.920789 (+ 260us) server_negotiation.cc:407] Connection header received
0808 23:27:55.920938 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:55.920941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:55.921037 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:55.921125 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:55.921817 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:55.922589 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:55.923744 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:55.924198 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:55.924301 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:55.927751 (+ 3450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.66:50843: BlockingRecv error: recv got EOF from 127.9.44.66:50843 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 08 23:27:56 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:56 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:27:56.373268 23852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:56.364742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.67:43177)
0808 23:27:56.365203 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:56.365220 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:56.365344 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:56.365744 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:56.365748 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:56.366223 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:56.366449 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:56.366465 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:56.367594 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:56.367598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:56.368754 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:56.368768 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:56.369439 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:56.369444 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:56.369576 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:56.370135 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:56.370151 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:56.373008 (+ 2857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":262,"threads_started":1}
W20260808 23:27:56.373572 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:56.364813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:43177 (local address 127.9.44.126:35037)
0808 23:27:56.365008 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:56.365016 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:56.365296 (+ 280us) server_negotiation.cc:407] Connection header received
0808 23:27:56.365472 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:56.365477 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:56.365566 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:56.365658 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:56.366605 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:56.367470 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:56.368882 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:56.369296 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:56.369389 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:56.373239 (+ 3850us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.67:43177: BlockingRecv error: recv got EOF from 127.9.44.67:43177 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 23:27:56 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:56 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:27:56.495209 23854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:56.487034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.65:39951)
0808 23:27:56.487533 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:56.487548 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:56.487697 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:56.488044 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:56.488048 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:56.488436 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:56.488583 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:56.488593 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:56.489600 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:56.489605 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:56.490573 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:56.490585 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:56.491267 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:56.491273 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:56.491445 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:56.492020 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:56.492038 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:56.494891 (+ 2853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":260,"threads_started":1}
W20260808 23:27:56.495353 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:56.487127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:39951 (local address 127.9.44.126:35037)
0808 23:27:56.487326 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:56.487333 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:56.487632 (+ 299us) server_negotiation.cc:407] Connection header received
0808 23:27:56.487825 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:56.487831 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:56.487900 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:56.487983 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:56.488726 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:56.489476 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:56.490713 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:56.491131 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:56.491244 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:56.495091 (+ 3847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.65:39951: BlockingRecv error: recv got EOF from 127.9.44.65:39951 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 23:27:56 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:56 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:27:56.936779 23855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:56.928677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.66:37133)
0808 23:27:56.929150 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:56.929167 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:56.929287 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:56.929574 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:56.929577 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:56.929961 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:56.930113 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:56.930123 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:56.931077 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:56.931082 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:56.932367 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:56.932386 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:56.933055 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:56.933062 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:56.933259 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:56.933821 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:56.933839 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:56.936530 (+ 2691us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":254,"threads_started":1}
W20260808 23:27:56.936944 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:56.928728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:37133 (local address 127.9.44.126:35037)
0808 23:27:56.928918 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:56.928925 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:56.929241 (+ 316us) server_negotiation.cc:407] Connection header received
0808 23:27:56.929376 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:56.929380 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:56.929455 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:56.929533 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:56.930219 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:56.930936 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:56.932500 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:56.932913 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:56.933019 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:56.936729 (+ 3710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.66:37133: BlockingRecv error: recv got EOF from 127.9.44.66:37133 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 23:27:57 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:57 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:27:57.382059 23856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:57.374148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.67:39819)
0808 23:27:57.374614 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:57.374631 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:57.374755 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:57.375032 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:57.375035 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:57.375396 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:57.375556 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:57.375566 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:57.376523 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:57.376526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:57.377476 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:57.377488 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:57.378073 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:57.378077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:57.378190 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:57.378726 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:57.378760 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:57.381746 (+ 2986us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":252,"threads_started":1}
W20260808 23:27:57.382251 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:57.374211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:39819 (local address 127.9.44.126:35037)
0808 23:27:57.374404 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:57.374411 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:57.374702 (+ 291us) server_negotiation.cc:407] Connection header received
0808 23:27:57.374845 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:57.374849 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:57.374925 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:57.375003 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:57.375688 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:57.376411 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:57.377581 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:57.377975 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:57.378059 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:57.381992 (+ 3933us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.67:39819: BlockingRecv error: recv got EOF from 127.9.44.67:39819 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 23:27:57 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:57 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:27:57.507536 23858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:57.496119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.65:54297)
0808 23:27:57.496605 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:57.496621 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:57.496730 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:57.496985 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:57.496988 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:57.497388 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:57.497560 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:57.497569 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:57.498458 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:57.498461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:57.499364 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:57.499375 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:57.500029 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:57.500033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:57.500155 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:57.504334 (+ 4179us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:57.504352 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:57.507197 (+ 2845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":242,"threads_started":1}
W20260808 23:27:57.507622 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:57.496194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:54297 (local address 127.9.44.126:35037)
0808 23:27:57.496408 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:57.496414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:57.496695 (+ 281us) server_negotiation.cc:407] Connection header received
0808 23:27:57.496817 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:57.496820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:57.496879 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:57.496957 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:57.497679 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:57.498355 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:57.499466 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:57.499925 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:57.500014 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:57.507418 (+ 7404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.65:54297: BlockingRecv error: recv got EOF from 127.9.44.65:54297 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 23:27:57 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:57 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:27:57.946105 23860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:57.937643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.66:50741)
0808 23:27:57.938152 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:57.938169 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:57.938286 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:57.938626 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:57.938630 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:57.939027 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:57.939229 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:57.939238 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:57.940416 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:57.940421 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:57.941621 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:57.941636 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:57.942374 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:57.942379 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:57.942505 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:57.943075 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:57.943093 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:57.945855 (+ 2762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":227,"threads_started":1}
W20260808 23:27:57.946238 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:57.937780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:50741 (local address 127.9.44.126:35037)
0808 23:27:57.937935 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:57.937940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:57.938234 (+ 294us) server_negotiation.cc:407] Connection header received
0808 23:27:57.938408 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:57.938412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:57.938490 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:57.938567 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:57.939335 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:57.940287 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:57.941772 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:57.942226 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:57.942328 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:57.946029 (+ 3701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.66:50741: BlockingRecv error: recv got EOF from 127.9.44.66:50741 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 23:27:58 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:58 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:27:58.391489 23861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:58.383014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.67:60761)
0808 23:27:58.383481 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:58.383497 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:58.383627 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:58.383988 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:58.383993 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:58.384451 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:58.384608 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:58.384616 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:58.385633 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:58.385637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:58.386556 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:58.386568 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:58.387287 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:58.387293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:58.387457 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:58.388007 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:58.388025 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:58.391145 (+ 3120us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":227,"threads_started":1}
W20260808 23:27:58.391626 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:58.383085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:60761 (local address 127.9.44.126:35037)
0808 23:27:58.383269 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:58.383275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:58.383583 (+ 308us) server_negotiation.cc:407] Connection header received
0808 23:27:58.383762 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:58.383767 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:58.383844 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:58.383938 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:58.384756 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:58.385495 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:58.386700 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:58.387146 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:58.387256 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:58.391405 (+ 4149us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.67:60761: BlockingRecv error: recv got EOF from 127.9.44.67:60761 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 23:27:58 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:58 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:27:58.516462 23862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:58.508382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.65:43781)
0808 23:27:58.508883 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:58.508899 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:58.509029 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:58.509345 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:58.509350 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:58.509758 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:58.509928 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:58.509938 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:58.510978 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:58.510982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:58.511968 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:58.511983 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:58.512701 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:58.512707 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:58.512891 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:58.513414 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:58.513432 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:58.516151 (+ 2719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":279,"threads_started":1}
W20260808 23:27:58.516657 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:58.508456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:43781 (local address 127.9.44.126:35037)
0808 23:27:58.508629 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:58.508636 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:58.508981 (+ 345us) server_negotiation.cc:407] Connection header received
0808 23:27:58.509137 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:58.509143 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:58.509212 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:58.509294 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:58.510078 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:58.510839 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:58.512109 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:58.512549 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:58.512697 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:58.516426 (+ 3729us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.65:43781: BlockingRecv error: recv got EOF from 127.9.44.65:43781 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 23:27:58 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:58 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:27:58.954759 23863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:58.946977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.66:46717)
0808 23:27:58.947503 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:58.947520 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:58.947634 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:58.947943 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:58.947946 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:58.948329 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:58.948501 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:58.948510 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:58.949412 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:58.949415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:58.950297 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:58.950308 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:58.950933 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:58.950937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:58.951093 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:58.951772 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:58.951792 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:58.954531 (+ 2739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":289,"threads_started":1}
W20260808 23:27:58.954924 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:58.947049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:46717 (local address 127.9.44.126:35037)
0808 23:27:58.947202 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:58.947208 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:58.947584 (+ 376us) server_negotiation.cc:407] Connection header received
0808 23:27:58.947764 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:58.947768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:58.947831 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:58.947911 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:58.948605 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:58.949306 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:58.950414 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:58.950809 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:58.950894 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:58.954704 (+ 3810us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.66:46717: BlockingRecv error: recv got EOF from 127.9.44.66:46717 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 23:27:59 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:59 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:27:59.400436 23865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:59.392340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.67:57235)
0808 23:27:59.392837 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:59.392851 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:59.392980 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:59.393289 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:59.393293 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:59.393720 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:59.393881 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:59.393889 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:59.394886 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:59.394890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:59.395863 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:59.395876 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:59.396592 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:59.396598 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:59.396766 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:59.397343 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:59.397360 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:59.400150 (+ 2790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":239,"threads_started":1}
W20260808 23:27:59.400609 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:59.392434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:57235 (local address 127.9.44.126:35037)
0808 23:27:59.392661 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:59.392667 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:59.392933 (+ 266us) server_negotiation.cc:407] Connection header received
0808 23:27:59.393093 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:59.393098 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:59.393165 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:59.393257 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:59.394022 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:59.394760 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:59.396000 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:59.396446 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:59.396553 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:59.400382 (+ 3829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.67:57235: BlockingRecv error: recv got EOF from 127.9.44.67:57235 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 23:27:59 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:59 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:27:59.525017 23866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:59.517389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.65:49501)
0808 23:27:59.518007 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:59.518023 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:59.518130 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:59.518390 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:59.518393 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:59.518789 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:59.518956 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:59.518965 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:59.519922 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:59.519926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:59.520849 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:59.520860 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:59.521484 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:59.521488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:59.521596 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:59.522167 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:59.522185 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:59.524766 (+ 2581us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":286,"threads_started":1}
W20260808 23:27:59.525141 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:59.517493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:49501 (local address 127.9.44.126:35037)
0808 23:27:59.517653 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:59.517658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:59.518085 (+ 427us) server_negotiation.cc:407] Connection header received
0808 23:27:59.518215 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:59.518218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:59.518279 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:59.518369 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:59.519064 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:59.519818 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:59.520949 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:59.521376 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:59.521463 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:59.524958 (+ 3495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.65:49501: BlockingRecv error: recv got EOF from 127.9.44.65:49501 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 23:27:59 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:27:59 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:27:59.965058 23867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:27:59.955544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.66:47471)
0808 23:27:59.956130 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0808 23:27:59.956147 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:27:59.956261 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:27:59.956545 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:27:59.956548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:27:59.956936 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:27:59.957114 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:59.957122 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:59.958545 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:59.958549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:59.959550 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:27:59.959564 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:27:59.960593 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:27:59.960598 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:27:59.960724 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:59.961417 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:27:59.961441 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:27:59.964805 (+ 3364us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":284,"threads_started":1}
W20260808 23:27:59.965292 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:27:59.955591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:47471 (local address 127.9.44.126:35037)
0808 23:27:59.955824 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 23:27:59.955832 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:27:59.956220 (+ 388us) server_negotiation.cc:407] Connection header received
0808 23:27:59.956362 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:27:59.956366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:27:59.956428 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:27:59.956504 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:27:59.957258 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:59.958419 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:59.959707 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:27:59.960450 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:27:59.960562 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:27:59.965037 (+ 4475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.66:47471: BlockingRecv error: recv got EOF from 127.9.44.66:47471 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 08 23:28:00 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:28:00 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:28:00.409101 23870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:00.401406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.67:37807)
0808 23:28:00.401974 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:00.401990 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:00.402105 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:00.402396 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:00.402399 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:00.402805 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:00.403001 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:00.403010 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:00.403958 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:00.403962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:00.404869 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:00.404882 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:00.405512 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:00.405515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:00.405620 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:00.406140 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:00.406156 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:00.408743 (+ 2587us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":242,"threads_started":1}
W20260808 23:28:00.409191 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:28:00.401445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:37807 (local address 127.9.44.126:35037)
0808 23:28:00.401603 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:00.401607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:00.402053 (+ 446us) server_negotiation.cc:407] Connection header received
0808 23:28:00.402207 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:00.402210 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:00.402290 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:00.402369 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:00.403108 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:00.403855 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:00.404978 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:00.405407 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:00.405491 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:00.408972 (+ 3481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.67:37807: BlockingRecv error: recv got EOF from 127.9.44.67:37807 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 23:28:00 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:28:00 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:28:00.533840 23872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:00.526041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.65:44339)
0808 23:28:00.526522 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:00.526539 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:00.526659 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:00.526917 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:00.526920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:00.527319 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:00.527474 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:00.527483 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:00.528477 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:00.528481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:00.529410 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:00.529423 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:00.530063 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:00.530067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:00.530189 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:00.530733 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:00.530751 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:00.533588 (+ 2837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":256,"threads_started":1}
W20260808 23:28:00.533968 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:28:00.526232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:44339 (local address 127.9.44.126:35037)
0808 23:28:00.526439 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:00.526445 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:00.526605 (+ 160us) server_negotiation.cc:407] Connection header received
0808 23:28:00.526746 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:00.526750 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:00.526812 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:00.526900 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:00.527589 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:00.528351 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:00.529518 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:00.529924 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:00.530009 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:00.533781 (+ 3772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.65:44339: BlockingRecv error: recv got EOF from 127.9.44.65:44339 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 08 23:28:00 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:28:00 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:28:00.973737 23873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:00.965885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.66:41095)
0808 23:28:00.966332 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:00.966349 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:00.966464 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:00.966721 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:00.966724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:00.967146 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:00.967307 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:00.967316 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:00.968427 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:00.968430 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:00.969327 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:00.969340 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:00.970008 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:00.970012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:00.970125 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:00.970650 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:00.970668 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:00.973510 (+ 2842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":219,"threads_started":1}
W20260808 23:28:00.973870 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:28:00.965973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:41095 (local address 127.9.44.126:35037)
0808 23:28:00.966181 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:00.966188 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:00.966423 (+ 235us) server_negotiation.cc:407] Connection header received
0808 23:28:00.966553 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:00.966556 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:00.966616 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:00.966692 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:00.967412 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:00.968325 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:00.969435 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:00.969872 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:00.969961 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:00.973670 (+ 3709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.66:41095: BlockingRecv error: recv got EOF from 127.9.44.66:41095 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 08 23:28:01 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:28:01 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:28:01.418459 23874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:01.410082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.67:44577)
0808 23:28:01.410810 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:01.410830 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:01.411130 (+ 300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:01.411464 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:01.411467 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:01.411904 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:01.412085 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:01.412096 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:01.413085 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:01.413089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:01.414088 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:01.414103 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:01.414877 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:01.414884 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:01.415033 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:01.415572 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:01.415589 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:01.418228 (+ 2639us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":247,"threads_started":1}
W20260808 23:28:01.418639 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:28:01.410179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:44577 (local address 127.9.44.126:35037)
0808 23:28:01.410399 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:01.410405 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:01.410922 (+ 517us) server_negotiation.cc:407] Connection header received
0808 23:28:01.411268 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:01.411272 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:01.411334 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:01.411414 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:01.412214 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:01.412954 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:01.414240 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:01.414694 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:01.414788 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:01.418413 (+ 3625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.67:44577: BlockingRecv error: recv got EOF from 127.9.44.67:44577 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 08 23:28:01 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:28:01 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:28:01.543046 23875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:01.534678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.65:33943)
0808 23:28:01.535353 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:01.535369 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:01.535497 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:01.535870 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:01.535875 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:01.536267 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:01.536489 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:01.536501 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:01.537687 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:01.537693 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:01.538637 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:01.538648 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:01.539339 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:01.539345 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:01.539515 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:01.540096 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:01.540114 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:01.542802 (+ 2688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":429,"threads_started":1}
W20260808 23:28:01.543185 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:28:01.534736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:33943 (local address 127.9.44.126:35037)
0808 23:28:01.534946 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:01.534953 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:01.535451 (+ 498us) server_negotiation.cc:407] Connection header received
0808 23:28:01.535610 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:01.535615 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:01.535702 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:01.535783 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:01.536624 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:01.537564 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:01.538778 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:01.539187 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:01.539301 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:01.542985 (+ 3684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.65:33943: BlockingRecv error: recv got EOF from 127.9.44.65:33943 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 23:28:01 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:28:01 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:28:01.982494 23879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:01.974579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.66:39179)
0808 23:28:01.975097 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:01.975113 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:01.975229 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:01.975485 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:01.975488 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:01.975923 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:01.976100 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:01.976110 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:01.977048 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:01.977051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:01.978007 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:01.978018 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:01.978677 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:01.978681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:01.978801 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:01.979330 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:01.979347 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:01.982164 (+ 2817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":270,"threads_started":1}
W20260808 23:28:01.982591 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:28:01.974608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:39179 (local address 127.9.44.126:35037)
0808 23:28:01.974755 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:01.974760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:01.975174 (+ 414us) server_negotiation.cc:407] Connection header received
0808 23:28:01.975320 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:01.975324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:01.975384 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:01.975455 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:01.976210 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:01.976945 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:01.978108 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:01.978555 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:01.978652 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:01.982385 (+ 3733us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.66:39179: BlockingRecv error: recv got EOF from 127.9.44.66:39179 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 23:28:02 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:28:02 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 23:28:02.427500 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:28:02.419372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:33737 (local address 127.9.44.126:35037)
0808 23:28:02.419587 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:02.419594 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:02.419952 (+ 358us) server_negotiation.cc:407] Connection header received
0808 23:28:02.420138 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:02.420141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:02.420219 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:02.420309 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:02.421191 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:02.421906 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:02.423348 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:02.423802 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:02.423895 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:02.427324 (+ 3429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.67:33737: BlockingRecv error: recv got EOF from 127.9.44.67:33737 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 23:28:02.427515 23882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:02.419325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.67:33737)
0808 23:28:02.419867 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:02.419886 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:02.420029 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:02.420351 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:02.420356 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:02.420876 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:02.421068 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:02.421080 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:02.422042 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:02.422048 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:02.423214 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:02.423228 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:02.423919 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:02.423923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:02.424034 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:02.424535 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:02.424551 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:02.427147 (+ 2596us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":228,"threads_started":1}
Aug 08 23:28:02 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:28:02 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:28:02.555279 23884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:02.543919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.65:48619)
0808 23:28:02.544397 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:02.544414 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:02.544524 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:02.544794 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:02.544797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:02.545176 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:02.545328 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:02.545338 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:02.546668 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:02.546673 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:02.547612 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:02.547625 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:02.548270 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:02.548274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:02.548398 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:02.552425 (+ 4027us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:02.552443 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:02.555052 (+ 2609us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":267,"threads_started":1}
W20260808 23:28:02.555490 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:28:02.543988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:48619 (local address 127.9.44.126:35037)
0808 23:28:02.544175 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:02.544182 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:02.544487 (+ 305us) server_negotiation.cc:407] Connection header received
0808 23:28:02.544611 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:02.544615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:02.544674 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:02.544766 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:02.545441 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:02.546550 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:02.547758 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:02.548160 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:02.548252 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:02.555229 (+ 6977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.65:48619: BlockingRecv error: recv got EOF from 127.9.44.65:48619 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 23:28:02 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:28:02 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:28:02.991151 23885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:02.983499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.66:42385)
0808 23:28:02.984011 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:02.984027 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:02.984137 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:02.984418 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:02.984422 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:02.984811 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:02.984965 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:02.984974 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:02.985931 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:02.985935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:02.986853 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:02.986864 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:02.987532 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:02.987536 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:02.987659 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:02.988238 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:02.988255 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:02.990920 (+ 2665us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":288,"threads_started":1}
W20260808 23:28:02.991289 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:28:02.983551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:42385 (local address 127.9.44.126:35037)
0808 23:28:02.983753 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:02.983761 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:02.984109 (+ 348us) server_negotiation.cc:407] Connection header received
0808 23:28:02.984228 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:02.984231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:02.984295 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:02.984382 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:02.985101 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:02.985818 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:02.986981 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:02.987420 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:02.987512 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:02.991097 (+ 3585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.66:42385: BlockingRecv error: recv got EOF from 127.9.44.66:42385 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 23:28:03 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:28:03 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:28:03.436403 23889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:03.428425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.67:35451)
0808 23:28:03.428938 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:03.428953 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:03.429058 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:03.429363 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:03.429366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:03.429742 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:03.429901 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:03.429909 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:03.431118 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:03.431124 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:03.432089 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:03.432103 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:03.432787 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:03.432792 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:03.432961 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:03.433489 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:03.433505 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:03.436098 (+ 2593us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":293,"threads_started":1}
W20260808 23:28:03.436548 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:28:03.428599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:35451 (local address 127.9.44.126:35037)
0808 23:28:03.428830 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:03.428837 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:03.429020 (+ 183us) server_negotiation.cc:407] Connection header received
0808 23:28:03.429151 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:03.429156 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:03.429233 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:03.429332 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:03.430048 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:03.430976 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:03.432226 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:03.432635 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:03.432744 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:03.436307 (+ 3563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.67:35451: BlockingRecv error: recv got EOF from 127.9.44.67:35451 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
Aug 08 23:28:03 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:28:03 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:28:03.564276 23891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:03.556119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.65:40501)
0808 23:28:03.556627 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:03.556643 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:03.556776 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:03.557094 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:03.557099 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:03.557533 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:03.557714 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:03.557723 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:03.558790 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:03.558794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:03.559748 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:03.559761 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:03.560493 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:03.560499 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:03.560680 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:03.561213 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:03.561231 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:03.564037 (+ 2806us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":274,"threads_started":1}
W20260808 23:28:03.564463 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:28:03.556232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:40501 (local address 127.9.44.126:35037)
0808 23:28:03.556436 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:03.556443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:03.556728 (+ 285us) server_negotiation.cc:407] Connection header received
0808 23:28:03.556893 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:03.556898 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:03.556960 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:03.557043 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:03.557885 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:03.558653 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:03.559886 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:03.560337 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:03.560456 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:03.564262 (+ 3806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.65:40501: BlockingRecv error: recv got EOF from 127.9.44.65:40501 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 23:28:03 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:28:03 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:28:04.000138 23893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:03.991969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.66:57955)
0808 23:28:03.992509 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:03.992527 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:03.992639 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:03.992903 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:03.992907 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:03.993326 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:03.993510 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:03.993519 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:03.994468 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:03.994474 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:03.995827 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:03.995842 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:03.996500 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:03.996504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:03.996622 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:03.997150 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:03.997167 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:03.999893 (+ 2726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":297,"threads_started":1}
W20260808 23:28:04.000284 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:28:03.992074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:57955 (local address 127.9.44.126:35037)
0808 23:28:03.992231 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:03.992236 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:03.992587 (+ 351us) server_negotiation.cc:407] Connection header received
0808 23:28:03.992723 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:03.992726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:03.992785 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:03.992861 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:03.993626 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:03.994339 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:03.995958 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:03.996387 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:03.996478 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:04.000067 (+ 3589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.66:57955: BlockingRecv error: recv got EOF from 127.9.44.66:57955 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 23:28:04 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:28:04 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:28:04.445384 23897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:04.437296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.67:41713)
0808 23:28:04.437791 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:04.437807 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:04.437915 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:04.438179 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:04.438182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:04.438598 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:04.438801 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:04.438811 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:04.440087 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:04.440110 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:04.441145 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:04.441160 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:04.441813 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:04.441817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:04.441955 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:04.442555 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:04.442573 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:04.445150 (+ 2577us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":248,"threads_started":1}
W20260808 23:28:04.445559 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:28:04.437395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:41713 (local address 127.9.44.126:35037)
0808 23:28:04.437584 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:04.437588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:04.437877 (+ 289us) server_negotiation.cc:407] Connection header received
0808 23:28:04.438016 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:04.438019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:04.438080 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:04.438151 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:04.438932 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:04.439910 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:04.441268 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:04.441660 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:04.441783 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:04.445338 (+ 3555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.67:41713: BlockingRecv error: recv got EOF from 127.9.44.67:41713 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 23:28:04 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:28:04 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:28:04.573328 23898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:04.565113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.65:38385)
0808 23:28:04.565624 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:04.565639 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:04.565759 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:04.566033 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:04.566036 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:04.566410 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:04.566567 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:04.566576 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:04.567544 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:04.567549 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:04.568515 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:04.568528 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:04.569303 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:04.569307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:04.569481 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:04.570142 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:04.570167 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:04.573095 (+ 2928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":282,"threads_started":1}
W20260808 23:28:04.573493 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:28:04.565173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:38385 (local address 127.9.44.126:35037)
0808 23:28:04.565364 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:04.565369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:04.565704 (+ 335us) server_negotiation.cc:407] Connection header received
0808 23:28:04.565845 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:04.565849 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:04.565915 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:04.565993 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:04.566674 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:04.567412 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:04.568640 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:04.569164 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:04.569297 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:04.573288 (+ 3991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.65:38385: BlockingRecv error: recv got EOF from 127.9.44.65:38385 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 08 23:28:05 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:28:05 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:28:05.009043 23899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:05.000992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.66:36997)
0808 23:28:05.001466 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:05.001484 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:05.001599 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:05.001897 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:05.001900 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:05.002300 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:05.002468 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:05.002476 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:05.003430 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:05.003433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:05.004390 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:05.004403 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:05.005027 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:05.005031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:05.005157 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:05.005749 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:05.005768 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:05.008789 (+ 3021us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":248,"threads_started":1}
W20260808 23:28:05.009169 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:28:05.001022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:36997 (local address 127.9.44.126:35037)
0808 23:28:05.001214 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:05.001221 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:05.001559 (+ 338us) server_negotiation.cc:407] Connection header received
0808 23:28:05.001689 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:05.001693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:05.001768 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:05.001846 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:05.002571 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:05.003323 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:05.004496 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:05.004926 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:05.005011 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:05.008967 (+ 3956us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.66:36997: BlockingRecv error: recv got EOF from 127.9.44.66:36997 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 23:28:05 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:28:05 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:28:05.454025 23900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:05.446176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.67:53971)
0808 23:28:05.446686 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:05.446706 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:05.446889 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:05.447292 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:05.447299 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:05.447823 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:05.447990 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:05.448000 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:05.448941 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:05.448944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:05.449840 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:05.449851 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:05.450469 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:05.450472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:05.450593 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:05.451128 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:05.451144 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:05.453803 (+ 2659us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":284,"threads_started":1}
W20260808 23:28:05.454312 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:28:05.446224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:53971 (local address 127.9.44.126:35037)
0808 23:28:05.446394 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:05.446399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:05.446786 (+ 387us) server_negotiation.cc:407] Connection header received
0808 23:28:05.447008 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:05.447013 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:05.447072 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:05.447159 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:05.448128 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:05.448823 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:05.449943 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:05.450367 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:05.450454 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:05.454042 (+ 3588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.67:53971: BlockingRecv error: recv got EOF from 127.9.44.67:53971 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 23:28:05 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:28:05 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:28:05.582046 23904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:05.574139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.65:45225)
0808 23:28:05.574680 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:05.574696 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:05.574813 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:05.575073 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:05.575077 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:05.575456 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:05.575613 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:05.575622 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:05.576600 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:05.576604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:05.577522 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:05.577532 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:05.578243 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:05.578247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:05.578369 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:05.578881 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:05.578898 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:05.581791 (+ 2893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":283,"threads_started":1}
W20260808 23:28:05.582142 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:28:05.574208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:45225 (local address 127.9.44.126:35037)
0808 23:28:05.574357 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:05.574361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:05.574764 (+ 403us) server_negotiation.cc:407] Connection header received
0808 23:28:05.574905 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:05.574909 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:05.574969 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:05.575046 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:05.575779 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:05.576497 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:05.577652 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:05.578103 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:05.578193 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:05.581955 (+ 3762us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.65:45225: BlockingRecv error: recv got EOF from 127.9.44.65:45225 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 23:28:06 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:28:06 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:28:06.017918 23905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:06.009982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.66:59651)
0808 23:28:06.010467 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:06.010483 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:06.010594 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:06.010851 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:06.010854 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:06.011229 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:06.011391 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:06.011403 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:06.012414 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:06.012418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:06.013406 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:06.013419 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:06.014134 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:06.014139 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:06.014256 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:06.014821 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:06.014844 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:06.017605 (+ 2761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":233,"threads_started":1}
W20260808 23:28:06.018033 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:28:06.010112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:59651 (local address 127.9.44.126:35037)
0808 23:28:06.010313 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:06.010319 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:06.010552 (+ 233us) server_negotiation.cc:407] Connection header received
0808 23:28:06.010682 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:06.010685 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:06.010749 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:06.010827 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:06.011522 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:06.012306 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:06.013521 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:06.013987 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:06.014079 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:06.017797 (+ 3718us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.66:59651: BlockingRecv error: recv got EOF from 127.9.44.66:59651 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 08 23:28:06 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:28:06 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:28:06.463645 23906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:06.455086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.67:45741)
0808 23:28:06.455598 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:06.455614 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:06.455757 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:06.456096 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:06.456101 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:06.456547 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:06.456713 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:06.456723 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:06.457747 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:06.457751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:06.459095 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:06.459109 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:06.459917 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:06.459923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:06.460132 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:06.460652 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:06.460669 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:06.463397 (+ 2728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":269,"threads_started":1}
W20260808 23:28:06.463840 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:28:06.455161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:45741 (local address 127.9.44.126:35037)
0808 23:28:06.455371 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:06.455376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:06.455690 (+ 314us) server_negotiation.cc:407] Connection header received
0808 23:28:06.455892 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:06.455897 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:06.455959 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:06.456042 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:06.456878 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:06.457603 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:06.459236 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:06.459746 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:06.459871 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:06.463580 (+ 3709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.67:45741: BlockingRecv error: recv got EOF from 127.9.44.67:45741 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 23:28:06 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:28:06 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:28:06.590411 23907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:06.582882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.65:47833)
0808 23:28:06.583322 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:06.583338 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:06.583456 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:06.583714 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:06.583717 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:06.584085 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:06.584262 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:06.584272 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:06.585237 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:06.585240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:06.586150 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:06.586163 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:06.586746 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:06.586750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:06.586874 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:06.587467 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:06.587485 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:06.590148 (+ 2663us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":224,"threads_started":1}
W20260808 23:28:06.590529 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:28:06.582958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:47833 (local address 127.9.44.126:35037)
0808 23:28:06.583279 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:06.583286 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:06.583409 (+ 123us) server_negotiation.cc:407] Connection header received
0808 23:28:06.583542 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:06.583545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:06.583607 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:06.583700 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:06.584387 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:06.585133 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:06.586254 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:06.586636 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:06.586719 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:06.590329 (+ 3610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.65:47833: BlockingRecv error: recv got EOF from 127.9.44.65:47833 (error 108)
Metrics: {"server-negotiator.queue_time_us":183}
Aug 08 23:28:07 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:28:07 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:28:07.026712 23909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:07.018821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.66:55879)
0808 23:28:07.019276 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:07.019290 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:07.019417 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:07.019763 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:07.019768 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:07.020151 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:07.020329 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:07.020339 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:07.021341 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:07.021345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:07.022289 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:07.022301 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:07.023003 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:07.023008 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:07.023182 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:07.023738 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:07.023755 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:07.026451 (+ 2696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":231,"threads_started":1}
W20260808 23:28:07.026855 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:28:07.018942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:55879 (local address 127.9.44.126:35037)
0808 23:28:07.019139 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:07.019145 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:07.019367 (+ 222us) server_negotiation.cc:407] Connection header received
0808 23:28:07.019525 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:07.019530 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:07.019594 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:07.019695 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:07.020477 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:07.021193 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:07.022426 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:07.022868 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:07.022975 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:07.026662 (+ 3687us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.66:55879: BlockingRecv error: recv got EOF from 127.9.44.66:55879 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 23:28:07 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:28:07 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.67@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:28:07.471901 23910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:07.464507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.67:34075)
0808 23:28:07.464958 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:07.464974 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:07.465081 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:07.465328 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:07.465331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:07.465662 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:07.465843 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:07.465852 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:07.466787 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:07.466790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:07.467732 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:07.467745 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:07.468363 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:07.468367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:07.468480 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:07.469007 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:07.469023 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:07.471643 (+ 2620us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":215,"threads_started":1}
W20260808 23:28:07.472077 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:28:07.464551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:34075 (local address 127.9.44.126:35037)
0808 23:28:07.464781 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:07.464787 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:07.465045 (+ 258us) server_negotiation.cc:407] Connection header received
0808 23:28:07.465169 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:07.465172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:07.465234 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:07.465310 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:07.465949 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:07.466684 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:07.467835 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:07.468262 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:07.468350 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:07.471862 (+ 3512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.67:34075: BlockingRecv error: recv got EOF from 127.9.44.67:34075 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 23:28:07 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:28:07 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.65@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:28:07.602113 23912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:07.591411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.65:43471)
0808 23:28:07.591919 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:07.591935 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:07.592065 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:07.592411 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:07.592416 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:07.592862 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:07.593056 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:07.593065 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:07.594075 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:07.594080 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:07.594998 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:07.595011 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:07.595734 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:07.595740 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:07.595898 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:07.599031 (+ 3133us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:07.599049 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:07.601764 (+ 2715us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":288,"threads_started":1}
W20260808 23:28:07.602211 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:28:07.591454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:43471 (local address 127.9.44.126:35037)
0808 23:28:07.591624 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:07.591631 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:07.592019 (+ 388us) server_negotiation.cc:407] Connection header received
0808 23:28:07.592184 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:07.592188 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:07.592260 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:07.592376 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:07.593203 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:07.593937 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:07.595141 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:07.595562 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:07.595697 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:07.601971 (+ 6274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.65:43471: BlockingRecv error: recv got EOF from 127.9.44.65:43471 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 23:28:08 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 23:28:08 dist-test-slave-6bbx krb5kdc[22010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.44.66@KRBTEST.COM for oryx/127.9.44.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 23:28:08.035598 23913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:08.027785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35037 (local address 127.9.44.66:40959)
0808 23:28:08.028276 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:08.028297 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:08.028456 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:08.028792 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:08.028795 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:08.029212 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:08.029407 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:08.029416 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:08.030350 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:08.030353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:08.031260 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:08.031271 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:08.031945 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:08.031949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:08.032064 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:08.032637 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:08.032654 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:08.035250 (+ 2596us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.44.126:35037: Server oryx/127.9.44.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":259,"threads_started":1}
W20260808 23:28:08.035744 23351 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:28:08.027819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:40959 (local address 127.9.44.126:35037)
0808 23:28:08.028006 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:08.028012 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:08.028381 (+ 369us) server_negotiation.cc:407] Connection header received
0808 23:28:08.028598 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:08.028603 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:08.028682 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:08.028768 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:08.029512 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:08.030242 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:08.031362 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:08.031842 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:08.031932 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:08.035473 (+ 3541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.44.66:40959: BlockingRecv error: recv got EOF from 127.9.44.66:40959 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 23:28:08.313325 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 23076
I20260808 23:28:08.325112 23916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:08.314353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41182 (local address 127.9.44.65:40607)
0808 23:28:08.314923 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:08.314931 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:08.314958 (+ 27us) server_negotiation.cc:407] Connection header received
0808 23:28:08.315033 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:08.315040 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:08.315303 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:08.315482 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:08.316976 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:08.318037 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:08.319174 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:08.319765 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:08.319859 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:08.322221 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:28:08.322230 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:28:08.322243 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:28:08.322274 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:28:08.324643 (+ 2369us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.44.65@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 23:28:08.324894 (+ 251us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41182: Request ticket server kudu/127.9.44.65@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":276,"threads_started":1}
W20260808 23:28:08.325202 23916 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41182: Request ticket server kudu/127.9.44.65@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 23:28:08.325218 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 23076: Not authorized: Client connection negotiation failed: client connection to 127.9.44.65:40607: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.44.65@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 23:28:08.325474 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 23076
I20260808 23:28:08.355199 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 23354
I20260808 23:28:08.366674 23920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:08.356197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59890 (local address 127.9.44.66:38953)
0808 23:28:08.356818 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:08.356826 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:08.356855 (+ 29us) server_negotiation.cc:407] Connection header received
0808 23:28:08.356914 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:08.356920 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:08.357135 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:08.357301 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:08.358587 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:08.359623 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:08.360846 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:08.361418 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:08.361532 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:08.363913 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:28:08.363921 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:28:08.363933 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:28:08.363976 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:28:08.366139 (+ 2163us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.44.66@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 23:28:08.366408 (+ 269us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59890: Request ticket server kudu/127.9.44.66@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":308,"threads_started":1}
W20260808 23:28:08.366717 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 23354: Not authorized: Client connection negotiation failed: client connection to 127.9.44.66:38953: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.44.66@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 23:28:08.366766 23920 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59890: Request ticket server kudu/127.9.44.66@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 23:28:08.366844 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 23354
I20260808 23:28:08.391414 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 23592
I20260808 23:28:08.402817 23922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:08.392322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53526 (local address 127.9.44.67:41061)
0808 23:28:08.392863 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:08.392870 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:08.392896 (+ 26us) server_negotiation.cc:407] Connection header received
0808 23:28:08.392958 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:08.392964 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:08.393175 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:08.393357 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:08.394649 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:08.395796 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:08.396962 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:08.397490 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:08.397602 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:08.400000 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:28:08.400008 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:28:08.400021 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:28:08.400064 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:28:08.402200 (+ 2136us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.44.67@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 23:28:08.402504 (+ 304us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53526: Request ticket server kudu/127.9.44.67@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":245,"threads_started":1}
W20260808 23:28:08.402846 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 23592: Not authorized: Client connection negotiation failed: client connection to 127.9.44.67:41061: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.44.67@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 23:28:08.402921 23922 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53526: Request ticket server kudu/127.9.44.67@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 23:28:08.402995 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 23592
I20260808 23:28:08.422797 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 22951
I20260808 23:28:08.423946 23064 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:28:08.543264 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 22951
2026-08-08T23:28:08Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21033 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T23:28:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:28:08Z Disabled control of system clock
I20260808 23:28:08.627403 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:46789
--webserver_interface=127.9.44.126
--webserver_port=0
--builtin_ntp_servers=127.9.44.84:40241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:46789
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 23:28:08.950686 23929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:28:08.950947 23929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:28:08.951064 23929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:28:08.958017 23929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:28:08.958113 23929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:28:08.958194 23929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:28:08.958266 23929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:28:08.958324 23929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:28:08.966995 23929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:40241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:46789
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:46789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:28:08.968159 23929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:28:08.969790 23929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:28:08.981177 23937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:28:08.981138 23938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:28:08.981099 23941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:28:08.982040 23929 server_base.cc:1034] running on GCE node
I20260808 23:28:08.982735 23929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:28:08.984017 23929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:28:08.985212 23929 hybrid_clock.cc:648] HybridClock initialized: now 1786231688985154 us; error 48 us; skew 500 ppm
I20260808 23:28:08.985525 23929 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:28:08.987635 23929 webserver.cc:460] Webserver started at http://127.9.44.126:33491/ using document root <none> and password file <none>
I20260808 23:28:08.988303 23929 fs_manager.cc:359] Metadata directory not provided
I20260808 23:28:08.988392 23929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:28:08.988670 23929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:28:08.991401 23929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "0a198f23dfca4d3ba07a2d1d8e87547f"
format_stamp: "Formatted at 2026-08-08 23:28:08 on dist-test-slave-6bbx"
server_key: "4929d4ffb44bf1d97bf98dcc2915ea76"
server_key_iv: "a7332946fd7e67e168f4249040d5a65a"
server_key_version: "encryptionkey@0"
I20260808 23:28:08.992158 23929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "0a198f23dfca4d3ba07a2d1d8e87547f"
format_stamp: "Formatted at 2026-08-08 23:28:08 on dist-test-slave-6bbx"
server_key: "4929d4ffb44bf1d97bf98dcc2915ea76"
server_key_iv: "a7332946fd7e67e168f4249040d5a65a"
server_key_version: "encryptionkey@0"
I20260808 23:28:08.996932 23929 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 23:28:09.000051 23953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:09.001267 23929 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260808 23:28:09.001428 23929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "0a198f23dfca4d3ba07a2d1d8e87547f"
format_stamp: "Formatted at 2026-08-08 23:28:08 on dist-test-slave-6bbx"
server_key: "4929d4ffb44bf1d97bf98dcc2915ea76"
server_key_iv: "a7332946fd7e67e168f4249040d5a65a"
server_key_version: "encryptionkey@0"
I20260808 23:28:09.001626 23929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:28:09.014642 23929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:28:09.015349 23929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:28:09.015606 23929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:28:09.032248 23929 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:46789
I20260808 23:28:09.032281 24042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:46789 every 8 connection(s)
I20260808 23:28:09.033406 23929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:28:09.035630 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 23929
I20260808 23:28:09.035837 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:28:09.036512 9393 external_mini_cluster.cc:1428] Setting key 6303fed59e61dbf351d3a7e6033fc05c
I20260808 23:28:09.037103 24044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:28:09.048650 24044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a198f23dfca4d3ba07a2d1d8e87547f: Bootstrap starting.
I20260808 23:28:09.049595 24050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:09.038408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41464 (local address 127.9.44.126:46789)
0808 23:28:09.039309 (+ 901us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:09.039318 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:09.039378 (+ 60us) server_negotiation.cc:407] Connection header received
0808 23:28:09.041405 (+ 2027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:09.041427 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:09.041859 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:09.042225 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:09.042805 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:09.044181 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:09.045516 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:09.046178 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:09.046373 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:09.047475 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:28:09.047482 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:28:09.047495 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:28:09.048019 (+ 524us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:28:09.048091 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:28:09.048116 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:28:09.048216 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:09.048462 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 23:28:09.048729 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":322,"threads_started":1}
I20260808 23:28:09.051930 24044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a198f23dfca4d3ba07a2d1d8e87547f: Neither blocks nor log segments found. Creating new log.
I20260808 23:28:09.052979 24044 log.cc:824] T 00000000000000000000000000000000 P 0a198f23dfca4d3ba07a2d1d8e87547f: Log is configured to *not* fsync() on all Append() calls
I20260808 23:28:09.055505 24044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a198f23dfca4d3ba07a2d1d8e87547f: No bootstrap required, opened a new log
I20260808 23:28:09.062389 24044 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a198f23dfca4d3ba07a2d1d8e87547f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a198f23dfca4d3ba07a2d1d8e87547f" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 46789 } }
I20260808 23:28:09.062671 24044 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a198f23dfca4d3ba07a2d1d8e87547f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:28:09.062749 24044 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a198f23dfca4d3ba07a2d1d8e87547f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a198f23dfca4d3ba07a2d1d8e87547f, State: Initialized, Role: FOLLOWER
I20260808 23:28:09.063228 24044 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a198f23dfca4d3ba07a2d1d8e87547f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a198f23dfca4d3ba07a2d1d8e87547f" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 46789 } }
I20260808 23:28:09.063421 24044 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a198f23dfca4d3ba07a2d1d8e87547f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:28:09.063519 24044 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a198f23dfca4d3ba07a2d1d8e87547f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:28:09.063635 24044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a198f23dfca4d3ba07a2d1d8e87547f [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:28:09.065559 24044 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a198f23dfca4d3ba07a2d1d8e87547f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a198f23dfca4d3ba07a2d1d8e87547f" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 46789 } }
I20260808 23:28:09.065980 24044 leader_election.cc:302] T 00000000000000000000000000000000 P 0a198f23dfca4d3ba07a2d1d8e87547f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a198f23dfca4d3ba07a2d1d8e87547f; no voters:
I20260808 23:28:09.066473 24044 leader_election.cc:288] T 00000000000000000000000000000000 P 0a198f23dfca4d3ba07a2d1d8e87547f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:28:09.066598 24053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a198f23dfca4d3ba07a2d1d8e87547f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:28:09.067237 24053 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a198f23dfca4d3ba07a2d1d8e87547f [term 1 LEADER]: Becoming Leader. State: Replica: 0a198f23dfca4d3ba07a2d1d8e87547f, State: Running, Role: LEADER
I20260808 23:28:09.067749 24053 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a198f23dfca4d3ba07a2d1d8e87547f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a198f23dfca4d3ba07a2d1d8e87547f" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 46789 } }
I20260808 23:28:09.068305 24044 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a198f23dfca4d3ba07a2d1d8e87547f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:28:09.070547 24056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a198f23dfca4d3ba07a2d1d8e87547f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a198f23dfca4d3ba07a2d1d8e87547f. Latest consensus state: current_term: 1 leader_uuid: "0a198f23dfca4d3ba07a2d1d8e87547f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a198f23dfca4d3ba07a2d1d8e87547f" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 46789 } } }
I20260808 23:28:09.070956 24056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a198f23dfca4d3ba07a2d1d8e87547f [sys.catalog]: This master's current role is: LEADER
I20260808 23:28:09.071790 24055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a198f23dfca4d3ba07a2d1d8e87547f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a198f23dfca4d3ba07a2d1d8e87547f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a198f23dfca4d3ba07a2d1d8e87547f" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 46789 } } }
I20260808 23:28:09.072208 24055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a198f23dfca4d3ba07a2d1d8e87547f [sys.catalog]: This master's current role is: LEADER
I20260808 23:28:09.074846 24066 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:28:09.080605 24066 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:28:09.089648 24066 catalog_manager.cc:1347] Generated new cluster ID: 99337e07d8c34cba928577168ec53a28
I20260808 23:28:09.089740 24066 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:28:09.103792 24066 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:28:09.105423 24066 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:28:09.114604 24066 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a198f23dfca4d3ba07a2d1d8e87547f: Generated new TSK 0
I20260808 23:28:09.115459 24066 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:28:09.123876 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:0
--local_ip_for_outbound_sockets=127.9.44.65
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:46789
--builtin_ntp_servers=127.9.44.84:40241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 23:28:09.449420 24078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:28:09.449672 24078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:28:09.449765 24078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:28:09.456589 24078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:28:09.456672 24078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:28:09.456799 24078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:28:09.465420 24078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:40241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:46789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:28:09.466465 24078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:28:09.468133 24078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:28:09.480497 24083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:28:09.480582 24084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:28:09.480535 24086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:28:09.481678 24078 server_base.cc:1034] running on GCE node
I20260808 23:28:09.482228 24078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:28:09.483029 24078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:28:09.484215 24078 hybrid_clock.cc:648] HybridClock initialized: now 1786231689484170 us; error 30 us; skew 500 ppm
I20260808 23:28:09.484475 24078 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:28:09.486454 24078 webserver.cc:460] Webserver started at http://127.9.44.65:33905/ using document root <none> and password file <none>
I20260808 23:28:09.487072 24078 fs_manager.cc:359] Metadata directory not provided
I20260808 23:28:09.487156 24078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:28:09.487416 24078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:28:09.490150 24078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/data/instance:
uuid: "8a23c290493f4cb98ac4ada3793dd81e"
format_stamp: "Formatted at 2026-08-08 23:28:09 on dist-test-slave-6bbx"
server_key: "d33a85d3b00ea201566980cbba297d6f"
server_key_iv: "a0a78ae2aa1695846c26cf22d2bb7035"
server_key_version: "encryptionkey@0"
I20260808 23:28:09.490870 24078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance:
uuid: "8a23c290493f4cb98ac4ada3793dd81e"
format_stamp: "Formatted at 2026-08-08 23:28:09 on dist-test-slave-6bbx"
server_key: "d33a85d3b00ea201566980cbba297d6f"
server_key_iv: "a0a78ae2aa1695846c26cf22d2bb7035"
server_key_version: "encryptionkey@0"
I20260808 23:28:09.495604 24078 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 23:28:09.498700 24097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:09.499948 24078 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 23:28:09.500098 24078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "8a23c290493f4cb98ac4ada3793dd81e"
format_stamp: "Formatted at 2026-08-08 23:28:09 on dist-test-slave-6bbx"
server_key: "d33a85d3b00ea201566980cbba297d6f"
server_key_iv: "a0a78ae2aa1695846c26cf22d2bb7035"
server_key_version: "encryptionkey@0"
I20260808 23:28:09.500289 24078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:28:09.513888 24078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:28:09.514566 24078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:28:09.514804 24078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:28:09.515632 24078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:28:09.517012 24078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:28:09.517098 24078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:09.517200 24078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:28:09.517277 24078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:09.547006 24078 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:36527
I20260808 23:28:09.547067 24284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:36527 every 8 connection(s)
I20260808 23:28:09.548270 24078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:28:09.558581 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 24078
I20260808 23:28:09.558745 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance
I20260808 23:28:09.559370 9393 external_mini_cluster.cc:1428] Setting key f910aff99a24882b7c43aae190035745
I20260808 23:28:09.560137 24291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:09.550706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:60369 (local address 127.9.44.126:46789)
0808 23:28:09.551118 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:09.551122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:09.552269 (+ 1147us) server_negotiation.cc:407] Connection header received
0808 23:28:09.553980 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:09.553986 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:09.554089 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:09.554206 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:09.555237 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:09.555992 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:09.557502 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:09.557917 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:09.558029 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:09.559301 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:28:09.559307 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:28:09.559310 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:28:09.559326 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:28:09.559373 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:28:09.559386 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:28:09.559462 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:09.559695 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 23:28:09.559790 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":205,"threads_started":1}
I20260808 23:28:09.561064 24292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:09.551227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:46789 (local address 127.9.44.65:60369)
0808 23:28:09.552123 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:09.552158 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:09.553716 (+ 1558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:09.554447 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:09.554456 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:09.554535 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:09.555069 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:09.555087 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:09.556140 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:09.556144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:09.557357 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:09.557371 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:09.558142 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:09.558151 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:09.558385 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:09.559008 (+ 623us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:28:09.559011 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:09.559049 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:28:09.559118 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:28:09.559480 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:28:09.559492 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:28:09.559499 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 23:28:09.559646 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 23:28:09.560019 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":277,"threads_started":1}
I20260808 23:28:09.561913 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:0
--local_ip_for_outbound_sockets=127.9.44.66
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:46789
--builtin_ntp_servers=127.9.44.84:40241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 23:28:09.562484 24288 heartbeater.cc:347] Connected to a master server at 127.9.44.126:46789
I20260808 23:28:09.562741 24288 heartbeater.cc:464] Registering TS with master...
I20260808 23:28:09.563436 24288 heartbeater.cc:511] Master 127.9.44.126:46789 requested a full tablet report, sending...
I20260808 23:28:09.565543 23984 ts_manager.cc:194] Registered new tserver with Master: 8a23c290493f4cb98ac4ada3793dd81e (127.9.44.65:36527)
I20260808 23:28:09.567344 23984 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.44.65:60369
W20260808 23:28:09.889384 24293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:28:09.889634 24293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:28:09.889714 24293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:28:09.896724 24293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:28:09.896813 24293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:28:09.896917 24293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:28:09.905658 24293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:40241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:46789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:28:09.906777 24293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:28:09.908665 24293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:28:09.920981 24303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:28:09.921008 24308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:28:09.920928 24305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:28:09.922144 24293 server_base.cc:1034] running on GCE node
I20260808 23:28:09.922752 24293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:28:09.923592 24293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:28:09.924818 24293 hybrid_clock.cc:648] HybridClock initialized: now 1786231689924757 us; error 50 us; skew 500 ppm
I20260808 23:28:09.925117 24293 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:28:09.927161 24293 webserver.cc:460] Webserver started at http://127.9.44.66:36893/ using document root <none> and password file <none>
I20260808 23:28:09.927850 24293 fs_manager.cc:359] Metadata directory not provided
I20260808 23:28:09.927956 24293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:28:09.928253 24293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:28:09.930907 24293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/data/instance:
uuid: "a33df04d7610483b9c83cc20cd90bc6e"
format_stamp: "Formatted at 2026-08-08 23:28:09 on dist-test-slave-6bbx"
server_key: "429a4bd3f56751b165774b2f7b2b66b8"
server_key_iv: "62fd604b194d42527cf5df19d7bc5feb"
server_key_version: "encryptionkey@0"
I20260808 23:28:09.931691 24293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance:
uuid: "a33df04d7610483b9c83cc20cd90bc6e"
format_stamp: "Formatted at 2026-08-08 23:28:09 on dist-test-slave-6bbx"
server_key: "429a4bd3f56751b165774b2f7b2b66b8"
server_key_iv: "62fd604b194d42527cf5df19d7bc5feb"
server_key_version: "encryptionkey@0"
I20260808 23:28:09.936489 24293 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 23:28:09.939703 24321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:09.940948 24293 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 23:28:09.941103 24293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "a33df04d7610483b9c83cc20cd90bc6e"
format_stamp: "Formatted at 2026-08-08 23:28:09 on dist-test-slave-6bbx"
server_key: "429a4bd3f56751b165774b2f7b2b66b8"
server_key_iv: "62fd604b194d42527cf5df19d7bc5feb"
server_key_version: "encryptionkey@0"
I20260808 23:28:09.941295 24293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:28:09.954196 24293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:28:09.954911 24293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:28:09.955154 24293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:28:09.956048 24293 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:28:09.957415 24293 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:28:09.957515 24293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:09.957617 24293 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:28:09.957697 24293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:09.987761 24293 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:41369
I20260808 23:28:09.987818 24500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:41369 every 8 connection(s)
I20260808 23:28:09.988938 24293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:28:09.997414 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 24293
I20260808 23:28:09.997576 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance
I20260808 23:28:09.998121 9393 external_mini_cluster.cc:1428] Setting key 68b061f9df4d7b9b4f5d610551014c92
I20260808 23:28:10.000814 24291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:09.991210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:50813 (local address 127.9.44.126:46789)
0808 23:28:09.991427 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:09.991433 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:09.992734 (+ 1301us) server_negotiation.cc:407] Connection header received
0808 23:28:09.994391 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:09.994395 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:09.994466 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:09.994554 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:09.995532 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:09.996290 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:09.998197 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:09.998602 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:09.998725 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:10.000159 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:28:10.000165 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:28:10.000168 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:28:10.000184 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:28:10.000217 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:28:10.000229 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:28:10.000302 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:10.000527 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 23:28:10.000598 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 23:28:10.001086 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:0
--local_ip_for_outbound_sockets=127.9.44.67
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:46789
--builtin_ntp_servers=127.9.44.84:40241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 23:28:10.002053 24503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:09.991716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:46789 (local address 127.9.44.66:50813)
0808 23:28:09.992588 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:09.992621 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:09.994140 (+ 1519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:09.994731 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:09.994738 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:09.994815 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:09.995373 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:09.995390 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:09.996406 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:09.996409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:09.998030 (+ 1621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:09.998046 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:09.998824 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:09.998832 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:09.999075 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:09.999797 (+ 722us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:28:09.999801 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:09.999849 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:28:09.999940 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:28:10.000327 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:28:10.000338 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:28:10.000346 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 23:28:10.000527 (+ 181us) client_negotiation.cc:240] Negotiation successful
0808 23:28:10.000888 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":274,"threads_started":1}
I20260808 23:28:10.003455 24501 heartbeater.cc:347] Connected to a master server at 127.9.44.126:46789
I20260808 23:28:10.003768 24501 heartbeater.cc:464] Registering TS with master...
I20260808 23:28:10.004644 24501 heartbeater.cc:511] Master 127.9.44.126:46789 requested a full tablet report, sending...
I20260808 23:28:10.006021 23984 ts_manager.cc:194] Registered new tserver with Master: a33df04d7610483b9c83cc20cd90bc6e (127.9.44.66:41369)
I20260808 23:28:10.006824 23984 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.44.66:50813
W20260808 23:28:10.330212 24505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:28:10.330482 24505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:28:10.330605 24505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:28:10.337525 24505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:28:10.337632 24505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:28:10.337783 24505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:28:10.346494 24505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:40241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:46789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:28:10.347609 24505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:28:10.349283 24505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:28:10.361297 24517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:28:10.361363 24512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:28:10.361340 24513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:28:10.362378 24505 server_base.cc:1034] running on GCE node
I20260808 23:28:10.362928 24505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:28:10.363773 24505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:28:10.364974 24505 hybrid_clock.cc:648] HybridClock initialized: now 1786231690364906 us; error 54 us; skew 500 ppm
I20260808 23:28:10.365267 24505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:28:10.367267 24505 webserver.cc:460] Webserver started at http://127.9.44.67:34003/ using document root <none> and password file <none>
I20260808 23:28:10.367981 24505 fs_manager.cc:359] Metadata directory not provided
I20260808 23:28:10.368085 24505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:28:10.368394 24505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:28:10.371068 24505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/data/instance:
uuid: "5d1152af67eb4fa898c9b75f7a88c105"
format_stamp: "Formatted at 2026-08-08 23:28:10 on dist-test-slave-6bbx"
server_key: "3780f0344b0c77cc63d156e7d8ef0054"
server_key_iv: "ce3b3a8f4e1065a1e79de82586568d1f"
server_key_version: "encryptionkey@0"
I20260808 23:28:10.371891 24505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance:
uuid: "5d1152af67eb4fa898c9b75f7a88c105"
format_stamp: "Formatted at 2026-08-08 23:28:10 on dist-test-slave-6bbx"
server_key: "3780f0344b0c77cc63d156e7d8ef0054"
server_key_iv: "ce3b3a8f4e1065a1e79de82586568d1f"
server_key_version: "encryptionkey@0"
I20260808 23:28:10.376736 24505 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 23:28:10.379900 24528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:10.381075 24505 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 23:28:10.381242 24505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "5d1152af67eb4fa898c9b75f7a88c105"
format_stamp: "Formatted at 2026-08-08 23:28:10 on dist-test-slave-6bbx"
server_key: "3780f0344b0c77cc63d156e7d8ef0054"
server_key_iv: "ce3b3a8f4e1065a1e79de82586568d1f"
server_key_version: "encryptionkey@0"
I20260808 23:28:10.381428 24505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:28:10.393841 24505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:28:10.394543 24505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:28:10.394779 24505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:28:10.395641 24505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:28:10.396986 24505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:28:10.397078 24505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:10.397181 24505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:28:10.397259 24505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:10.426724 24505 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:39213
I20260808 23:28:10.426782 24737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:39213 every 8 connection(s)
I20260808 23:28:10.427925 24505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:28:10.436283 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 24505
I20260808 23:28:10.436456 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance
I20260808 23:28:10.436991 9393 external_mini_cluster.cc:1428] Setting key 1daada1e61265de649fb7ccdf2c52a7e
I20260808 23:28:10.440115 24291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:10.430248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:55647 (local address 127.9.44.126:46789)
0808 23:28:10.430424 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:10.430429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:10.431731 (+ 1302us) server_negotiation.cc:407] Connection header received
0808 23:28:10.433492 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:10.433496 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:10.433566 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:10.433653 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:10.434593 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:10.435325 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:10.437367 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:10.437793 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:10.437934 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:10.439314 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:28:10.439321 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:28:10.439325 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:28:10.439342 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:28:10.439376 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:28:10.439388 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:28:10.439462 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:10.439701 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 23:28:10.439797 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 23:28:10.440994 24744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:10.430695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:46789 (local address 127.9.44.67:55647)
0808 23:28:10.431562 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:10.431596 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:10.433262 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:10.433841 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:10.433849 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:10.433929 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:10.434458 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:10.434476 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:10.435451 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:10.435454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:10.437218 (+ 1764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:10.437233 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:10.438022 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:10.438030 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:10.438286 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:10.438985 (+ 699us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:28:10.438989 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:10.439038 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:28:10.439129 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:28:10.439472 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:28:10.439484 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:28:10.439492 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 23:28:10.439651 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 23:28:10.440037 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":261,"threads_started":1}
I20260808 23:28:10.442097 24741 heartbeater.cc:347] Connected to a master server at 127.9.44.126:46789
I20260808 23:28:10.442374 24741 heartbeater.cc:464] Registering TS with master...
I20260808 23:28:10.443076 24741 heartbeater.cc:511] Master 127.9.44.126:46789 requested a full tablet report, sending...
I20260808 23:28:10.444417 23984 ts_manager.cc:194] Registered new tserver with Master: 5d1152af67eb4fa898c9b75f7a88c105 (127.9.44.67:39213)
I20260808 23:28:10.445192 23984 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.44.67:55647
I20260808 23:28:10.452034 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 23:28:10.457912 24291 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:28:10.456322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41472 (local address 127.9.44.126:46789)
0808 23:28:10.456532 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:10.456538 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:10.456977 (+ 439us) server_negotiation.cc:407] Connection header received
0808 23:28:10.457131 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:10.457134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:10.457216 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:10.457296 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:10.457700 (+ 404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41472: BlockingRecv error: recv got EOF from 127.0.0.1:41472 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260808 23:28:10.458832 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 24078
I20260808 23:28:10.465807 24754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:10.459632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39844 (local address 127.9.44.65:36527)
0808 23:28:10.460242 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:10.460256 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:10.460280 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:28:10.460340 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:10.460345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:10.460518 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:10.460634 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:10.460974 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:10.461931 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:10.463078 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:10.463587 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:10.463711 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:10.464569 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:28:10.464577 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:28:10.464588 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:28:10.465105 (+ 517us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:28:10.465154 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:28:10.465173 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:28:10.465260 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:10.465336 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 23:28:10.465476 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":296,"threads_started":1}
I20260808 23:28:10.466878 24277 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:28:10.545572 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 24078
I20260808 23:28:10.566772 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 24293
I20260808 23:28:10.574285 24757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:10.567648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59860 (local address 127.9.44.66:41369)
0808 23:28:10.568202 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:10.568211 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:10.568240 (+ 29us) server_negotiation.cc:407] Connection header received
0808 23:28:10.568321 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:10.568327 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:10.568567 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:10.568702 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:10.569063 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:10.570110 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:10.571295 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:10.571920 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:10.572013 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:10.573032 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:28:10.573040 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:28:10.573052 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:28:10.573613 (+ 561us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:28:10.573664 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:28:10.573683 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:28:10.573776 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:10.573870 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 23:28:10.573992 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":249,"threads_started":1}
I20260808 23:28:10.575429 24490 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:28:10.654491 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 24293
I20260808 23:28:10.679245 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 24505
I20260808 23:28:10.686951 24760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:10.680243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58684 (local address 127.9.44.67:39213)
0808 23:28:10.680839 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:10.680845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:10.680875 (+ 30us) server_negotiation.cc:407] Connection header received
0808 23:28:10.680948 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:10.680954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:10.681157 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:10.681285 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:10.681666 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:10.682716 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:10.684032 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:10.684584 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:10.684677 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:10.685640 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:28:10.685648 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:28:10.685661 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:28:10.686266 (+ 605us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:28:10.686313 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:28:10.686339 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:28:10.686430 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:10.686517 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 23:28:10.686638 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":294,"threads_started":1}
I20260808 23:28:10.688160 24727 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:28:10.767746 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 24505
I20260808 23:28:10.790911 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 23929
I20260808 23:28:10.792026 24036 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:28:10.889983 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 23929
2026-08-08T23:28:10Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2341 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T23:28:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:28:10Z Disabled control of system clock
I20260808 23:28:10.969605 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:34517
--webserver_interface=127.9.44.126
--webserver_port=0
--builtin_ntp_servers=127.9.44.84:41609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:34517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 23:28:11.288358 24768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:28:11.288600 24768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:28:11.288686 24768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:28:11.295433 24768 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:28:11.295526 24768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:28:11.295569 24768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:28:11.295622 24768 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:28:11.295680 24768 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:28:11.304245 24768 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:41609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:34517
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:34517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:28:11.305285 24768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:28:11.306926 24768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:28:11.317679 24784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:28:11.317735 24782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:28:11.317713 24779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:28:11.318734 24768 server_base.cc:1034] running on GCE node
I20260808 23:28:11.319427 24768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:28:11.320693 24768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:28:11.321878 24768 hybrid_clock.cc:648] HybridClock initialized: now 1786231691321831 us; error 41 us; skew 500 ppm
I20260808 23:28:11.322198 24768 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:28:11.326817 24768 webserver.cc:460] Webserver started at http://127.9.44.126:36405/ using document root <none> and password file <none>
I20260808 23:28:11.327453 24768 fs_manager.cc:359] Metadata directory not provided
I20260808 23:28:11.327548 24768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:28:11.327905 24768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:28:11.330608 24768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "68b610702c3746c0bb6b729bb88b6111"
format_stamp: "Formatted at 2026-08-08 23:28:11 on dist-test-slave-6bbx"
server_key: "2581fe1ebd9afda4eff3fc159fa4a628"
server_key_iv: "df3c2bae0f8ed571a5319423929a822e"
server_key_version: "encryptionkey@0"
I20260808 23:28:11.331346 24768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "68b610702c3746c0bb6b729bb88b6111"
format_stamp: "Formatted at 2026-08-08 23:28:11 on dist-test-slave-6bbx"
server_key: "2581fe1ebd9afda4eff3fc159fa4a628"
server_key_iv: "df3c2bae0f8ed571a5319423929a822e"
server_key_version: "encryptionkey@0"
I20260808 23:28:11.336059 24768 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260808 23:28:11.339213 24795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:11.340462 24768 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:28:11.340619 24768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "68b610702c3746c0bb6b729bb88b6111"
format_stamp: "Formatted at 2026-08-08 23:28:11 on dist-test-slave-6bbx"
server_key: "2581fe1ebd9afda4eff3fc159fa4a628"
server_key_iv: "df3c2bae0f8ed571a5319423929a822e"
server_key_version: "encryptionkey@0"
I20260808 23:28:11.340813 24768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:28:11.346179 24768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:28:11.346853 24768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:28:11.347114 24768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:28:11.363518 24768 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:34517
I20260808 23:28:11.363543 24889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:34517 every 8 connection(s)
I20260808 23:28:11.365106 24768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:28:11.369300 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 24768
I20260808 23:28:11.369515 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:28:11.369827 24893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:28:11.370340 9393 external_mini_cluster.cc:1428] Setting key 0fabd43497b0d78ec5d9d63fb58e8c02
I20260808 23:28:11.382545 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:11.373645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40672 (local address 127.9.44.126:34517)
0808 23:28:11.375045 (+ 1400us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:11.375059 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:11.376277 (+ 1218us) server_negotiation.cc:407] Connection header received
0808 23:28:11.377934 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:11.377968 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:11.378497 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:11.378935 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:11.380435 (+ 1500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:28:11.380444 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:28:11.380460 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:28:11.380481 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:28:11.380564 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:28:11.380587 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:28:11.380796 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:11.381083 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 23:28:11.381340 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":867,"thread_start_us":611,"threads_started":1}
I20260808 23:28:11.385859 24893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68b610702c3746c0bb6b729bb88b6111: Bootstrap starting.
I20260808 23:28:11.388991 24893 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68b610702c3746c0bb6b729bb88b6111: Neither blocks nor log segments found. Creating new log.
I20260808 23:28:11.390137 24893 log.cc:824] T 00000000000000000000000000000000 P 68b610702c3746c0bb6b729bb88b6111: Log is configured to *not* fsync() on all Append() calls
I20260808 23:28:11.393324 24893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68b610702c3746c0bb6b729bb88b6111: No bootstrap required, opened a new log
I20260808 23:28:11.402251 24893 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68b610702c3746c0bb6b729bb88b6111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b610702c3746c0bb6b729bb88b6111" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 34517 } }
I20260808 23:28:11.402688 24893 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68b610702c3746c0bb6b729bb88b6111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:28:11.402787 24893 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68b610702c3746c0bb6b729bb88b6111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68b610702c3746c0bb6b729bb88b6111, State: Initialized, Role: FOLLOWER
I20260808 23:28:11.403393 24893 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68b610702c3746c0bb6b729bb88b6111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b610702c3746c0bb6b729bb88b6111" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 34517 } }
I20260808 23:28:11.403658 24893 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68b610702c3746c0bb6b729bb88b6111 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:28:11.403827 24893 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68b610702c3746c0bb6b729bb88b6111 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:28:11.403993 24893 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68b610702c3746c0bb6b729bb88b6111 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:28:11.405993 24893 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68b610702c3746c0bb6b729bb88b6111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b610702c3746c0bb6b729bb88b6111" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 34517 } }
I20260808 23:28:11.406474 24893 leader_election.cc:302] T 00000000000000000000000000000000 P 68b610702c3746c0bb6b729bb88b6111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68b610702c3746c0bb6b729bb88b6111; no voters:
I20260808 23:28:11.407044 24893 leader_election.cc:288] T 00000000000000000000000000000000 P 68b610702c3746c0bb6b729bb88b6111 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:28:11.407193 24902 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68b610702c3746c0bb6b729bb88b6111 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:28:11.407888 24902 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68b610702c3746c0bb6b729bb88b6111 [term 1 LEADER]: Becoming Leader. State: Replica: 68b610702c3746c0bb6b729bb88b6111, State: Running, Role: LEADER
I20260808 23:28:11.408439 24902 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68b610702c3746c0bb6b729bb88b6111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b610702c3746c0bb6b729bb88b6111" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 34517 } }
I20260808 23:28:11.409108 24893 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68b610702c3746c0bb6b729bb88b6111 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:28:11.411441 24903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68b610702c3746c0bb6b729bb88b6111 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68b610702c3746c0bb6b729bb88b6111" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b610702c3746c0bb6b729bb88b6111" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 34517 } } }
I20260808 23:28:11.411940 24903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68b610702c3746c0bb6b729bb88b6111 [sys.catalog]: This master's current role is: LEADER
I20260808 23:28:11.412788 24905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68b610702c3746c0bb6b729bb88b6111 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68b610702c3746c0bb6b729bb88b6111. Latest consensus state: current_term: 1 leader_uuid: "68b610702c3746c0bb6b729bb88b6111" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b610702c3746c0bb6b729bb88b6111" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 34517 } } }
I20260808 23:28:11.413213 24905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68b610702c3746c0bb6b729bb88b6111 [sys.catalog]: This master's current role is: LEADER
I20260808 23:28:11.420625 24910 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:28:11.424721 24910 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:28:11.432112 24910 catalog_manager.cc:1347] Generated new cluster ID: 6060447e815b4d1fb88df228ff508638
I20260808 23:28:11.432219 24910 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:28:11.442652 24910 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:28:11.443020 24910 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:28:11.451310 24910 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68b610702c3746c0bb6b729bb88b6111: Generated new TSK 0
I20260808 23:28:11.452028 24910 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:28:11.458405 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:0
--local_ip_for_outbound_sockets=127.9.44.65
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:34517
--builtin_ntp_servers=127.9.44.84:41609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 23:28:11.781234 24942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:28:11.781503 24942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:28:11.781631 24942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:28:11.788522 24942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:28:11.788604 24942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:28:11.788738 24942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:28:11.797176 24942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:41609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:34517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:28:11.798274 24942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:28:11.799906 24942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:28:11.812029 24950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:28:11.811975 24955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:28:11.812000 24958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:28:11.813023 24942 server_base.cc:1034] running on GCE node
I20260808 23:28:11.813587 24942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:28:11.814456 24942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:28:11.815641 24942 hybrid_clock.cc:648] HybridClock initialized: now 1786231691815592 us; error 42 us; skew 500 ppm
I20260808 23:28:11.815953 24942 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:28:11.817953 24942 webserver.cc:460] Webserver started at http://127.9.44.65:34873/ using document root <none> and password file <none>
I20260808 23:28:11.818579 24942 fs_manager.cc:359] Metadata directory not provided
I20260808 23:28:11.818670 24942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:28:11.818970 24942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:28:11.821625 24942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/data/instance:
uuid: "3c1bb3483f3644de87df480c1bdcf483"
format_stamp: "Formatted at 2026-08-08 23:28:11 on dist-test-slave-6bbx"
server_key: "98ec0baf3b259a3f1598a238dff141a6"
server_key_iv: "29e3996b2092ee6081ba75f8ebbee41f"
server_key_version: "encryptionkey@0"
I20260808 23:28:11.822395 24942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance:
uuid: "3c1bb3483f3644de87df480c1bdcf483"
format_stamp: "Formatted at 2026-08-08 23:28:11 on dist-test-slave-6bbx"
server_key: "98ec0baf3b259a3f1598a238dff141a6"
server_key_iv: "29e3996b2092ee6081ba75f8ebbee41f"
server_key_version: "encryptionkey@0"
I20260808 23:28:11.827203 24942 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260808 23:28:11.830513 24965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:11.831776 24942 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:28:11.831924 24942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "3c1bb3483f3644de87df480c1bdcf483"
format_stamp: "Formatted at 2026-08-08 23:28:11 on dist-test-slave-6bbx"
server_key: "98ec0baf3b259a3f1598a238dff141a6"
server_key_iv: "29e3996b2092ee6081ba75f8ebbee41f"
server_key_version: "encryptionkey@0"
I20260808 23:28:11.832116 24942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:28:11.837491 24942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:28:11.838192 24942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:28:11.838434 24942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:28:11.839381 24942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:28:11.840711 24942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:28:11.840797 24942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:11.840899 24942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:28:11.840976 24942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:11.870895 24942 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:36357
I20260808 23:28:11.870955 25142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:36357 every 8 connection(s)
I20260808 23:28:11.872145 24942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:28:11.875415 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 24942
I20260808 23:28:11.875607 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance
I20260808 23:28:11.876243 9393 external_mini_cluster.cc:1428] Setting key b2c62185110fb0153fb28812f5db6b8c
I20260808 23:28:11.879027 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:0
--local_ip_for_outbound_sockets=127.9.44.66
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:34517
--builtin_ntp_servers=127.9.44.84:41609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 23:28:11.880105 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:11.874539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:36231 (local address 127.9.44.126:34517)
0808 23:28:11.874740 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:11.874747 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:11.876235 (+ 1488us) server_negotiation.cc:407] Connection header received
0808 23:28:11.878020 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:11.878024 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:11.878074 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:11.878175 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:11.879423 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:28:11.879429 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:28:11.879433 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:28:11.879452 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:28:11.879490 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:28:11.879504 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:28:11.879618 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:11.879780 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 23:28:11.879843 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 23:28:11.880868 25146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:11.875037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:34517 (local address 127.9.44.65:36231)
0808 23:28:11.876082 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:11.876136 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:11.877746 (+ 1610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:11.878373 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:11.878381 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:11.878465 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:11.879113 (+ 648us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:28:11.879115 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:11.879155 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:28:11.879258 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:28:11.879587 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:28:11.879592 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:28:11.879597 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:28:11.879836 (+ 239us) client_negotiation.cc:240] Negotiation successful
0808 23:28:11.880017 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":326,"threads_started":1}
I20260808 23:28:11.882308 25144 heartbeater.cc:347] Connected to a master server at 127.9.44.126:34517
I20260808 23:28:11.882589 25144 heartbeater.cc:464] Registering TS with master...
I20260808 23:28:11.883200 25144 heartbeater.cc:511] Master 127.9.44.126:34517 requested a full tablet report, sending...
I20260808 23:28:11.885313 24830 ts_manager.cc:194] Registered new tserver with Master: 3c1bb3483f3644de87df480c1bdcf483 (127.9.44.65:36357)
W20260808 23:28:12.206802 25148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:28:12.207050 25148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:28:12.207145 25148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:28:12.214318 25148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:28:12.214407 25148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:28:12.214538 25148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:28:12.223104 25148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:41609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:34517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:28:12.224277 25148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:28:12.225967 25148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:28:12.238481 25155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:28:12.238497 25157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:28:12.238639 25163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:28:12.239257 25148 server_base.cc:1034] running on GCE node
I20260808 23:28:12.239849 25148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:28:12.240715 25148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:28:12.241911 25148 hybrid_clock.cc:648] HybridClock initialized: now 1786231692241856 us; error 51 us; skew 500 ppm
I20260808 23:28:12.242211 25148 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:28:12.244371 25148 webserver.cc:460] Webserver started at http://127.9.44.66:38039/ using document root <none> and password file <none>
I20260808 23:28:12.245077 25148 fs_manager.cc:359] Metadata directory not provided
I20260808 23:28:12.245178 25148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:28:12.245528 25148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:28:12.248270 25148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/data/instance:
uuid: "25f5ff5e39db4ec5ac3e7dc4f4d187f4"
format_stamp: "Formatted at 2026-08-08 23:28:12 on dist-test-slave-6bbx"
server_key: "3002eead5b6df76db682efce9670c75c"
server_key_iv: "6cdca9e5c20c428b595d129220403ea4"
server_key_version: "encryptionkey@0"
I20260808 23:28:12.249079 25148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance:
uuid: "25f5ff5e39db4ec5ac3e7dc4f4d187f4"
format_stamp: "Formatted at 2026-08-08 23:28:12 on dist-test-slave-6bbx"
server_key: "3002eead5b6df76db682efce9670c75c"
server_key_iv: "6cdca9e5c20c428b595d129220403ea4"
server_key_version: "encryptionkey@0"
I20260808 23:28:12.253834 25148 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 23:28:12.257053 25171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:12.258199 25148 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 23:28:12.258358 25148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "25f5ff5e39db4ec5ac3e7dc4f4d187f4"
format_stamp: "Formatted at 2026-08-08 23:28:12 on dist-test-slave-6bbx"
server_key: "3002eead5b6df76db682efce9670c75c"
server_key_iv: "6cdca9e5c20c428b595d129220403ea4"
server_key_version: "encryptionkey@0"
I20260808 23:28:12.258539 25148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:28:12.263813 25148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:28:12.264508 25148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:28:12.264739 25148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:28:12.265560 25148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:28:12.266829 25148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:28:12.266923 25148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:12.267023 25148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:28:12.267091 25148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:12.295735 25148 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:44117
I20260808 23:28:12.295799 25368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:44117 every 8 connection(s)
I20260808 23:28:12.296921 25148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:28:12.304509 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:12.299291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:43771 (local address 127.9.44.126:34517)
0808 23:28:12.299479 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:12.299484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:12.300788 (+ 1304us) server_negotiation.cc:407] Connection header received
0808 23:28:12.302426 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:12.302432 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:12.302495 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:12.302585 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:12.303774 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:28:12.303780 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:28:12.303785 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:28:12.303803 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:28:12.303837 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:28:12.303849 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:28:12.303961 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:12.304152 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 23:28:12.304221 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 23:28:12.305142 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 25148
I20260808 23:28:12.305305 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance
I20260808 23:28:12.305322 25372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:12.299772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:34517 (local address 127.9.44.66:43771)
0808 23:28:12.300642 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:12.300680 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:12.302167 (+ 1487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:12.302809 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:12.302815 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:12.302900 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:12.303475 (+ 575us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:28:12.303478 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:12.303514 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:28:12.303595 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:28:12.303947 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:28:12.303957 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:28:12.303965 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 23:28:12.304214 (+ 249us) client_negotiation.cc:240] Negotiation successful
0808 23:28:12.304460 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":265,"threads_started":1}
I20260808 23:28:12.305845 9393 external_mini_cluster.cc:1428] Setting key 1a28c4877147dd479ca8c5e4bc5aed76
I20260808 23:28:12.306633 25370 heartbeater.cc:347] Connected to a master server at 127.9.44.126:34517
I20260808 23:28:12.306910 25370 heartbeater.cc:464] Registering TS with master...
I20260808 23:28:12.307441 25370 heartbeater.cc:511] Master 127.9.44.126:34517 requested a full tablet report, sending...
I20260808 23:28:12.308538 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:0
--local_ip_for_outbound_sockets=127.9.44.67
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:34517
--builtin_ntp_servers=127.9.44.84:41609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 23:28:12.308907 24830 ts_manager.cc:194] Registered new tserver with Master: 25f5ff5e39db4ec5ac3e7dc4f4d187f4 (127.9.44.66:44117)
W20260808 23:28:12.637257 25376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:28:12.637516 25376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:28:12.637645 25376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:28:12.644739 25376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:28:12.644842 25376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:28:12.644992 25376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:28:12.653612 25376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:41609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:34517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:28:12.654731 25376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:28:12.656422 25376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:28:12.668754 25390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:28:12.668797 25387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:28:12.668994 25386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:28:12.669584 25376 server_base.cc:1034] running on GCE node
I20260808 23:28:12.670130 25376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:28:12.670959 25376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:28:12.672163 25376 hybrid_clock.cc:648] HybridClock initialized: now 1786231692672106 us; error 42 us; skew 500 ppm
I20260808 23:28:12.672446 25376 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:28:12.677855 25376 webserver.cc:460] Webserver started at http://127.9.44.67:36813/ using document root <none> and password file <none>
I20260808 23:28:12.678485 25376 fs_manager.cc:359] Metadata directory not provided
I20260808 23:28:12.678586 25376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:28:12.678898 25376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:28:12.681571 25376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/data/instance:
uuid: "445af285092a4996bee75ae1cd6010ec"
format_stamp: "Formatted at 2026-08-08 23:28:12 on dist-test-slave-6bbx"
server_key: "96eb10dfcd497d9b9cd5052b9b1d0714"
server_key_iv: "b34ac29daac8d1b2dac49079c0d29bd8"
server_key_version: "encryptionkey@0"
I20260808 23:28:12.682336 25376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance:
uuid: "445af285092a4996bee75ae1cd6010ec"
format_stamp: "Formatted at 2026-08-08 23:28:12 on dist-test-slave-6bbx"
server_key: "96eb10dfcd497d9b9cd5052b9b1d0714"
server_key_iv: "b34ac29daac8d1b2dac49079c0d29bd8"
server_key_version: "encryptionkey@0"
I20260808 23:28:12.687152 25376 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260808 23:28:12.690394 25399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:12.691563 25376 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 23:28:12.691771 25376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "445af285092a4996bee75ae1cd6010ec"
format_stamp: "Formatted at 2026-08-08 23:28:12 on dist-test-slave-6bbx"
server_key: "96eb10dfcd497d9b9cd5052b9b1d0714"
server_key_iv: "b34ac29daac8d1b2dac49079c0d29bd8"
server_key_version: "encryptionkey@0"
I20260808 23:28:12.691974 25376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:28:12.697170 25376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:28:12.697896 25376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:28:12.698135 25376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:28:12.698975 25376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:28:12.700332 25376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:28:12.700428 25376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:12.700536 25376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:28:12.700609 25376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:12.729875 25376 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:45265
I20260808 23:28:12.729931 25586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:45265 every 8 connection(s)
I20260808 23:28:12.731070 25376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:28:12.732769 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 25376
I20260808 23:28:12.732956 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance
I20260808 23:28:12.733369 9393 external_mini_cluster.cc:1428] Setting key bcc13af5e76357b1b6ff2f01b1372d3e
I20260808 23:28:12.739073 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:12.733644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:47281 (local address 127.9.44.126:34517)
0808 23:28:12.733882 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:12.733889 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:12.735315 (+ 1426us) server_negotiation.cc:407] Connection header received
0808 23:28:12.737109 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:12.737116 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:12.737201 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:12.737320 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:12.738552 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:28:12.738556 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:28:12.738559 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:28:12.738575 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:28:12.738607 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:28:12.738619 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:28:12.738712 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:12.738838 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 23:28:12.738887 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 23:28:12.739953 25591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:12.734158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:34517 (local address 127.9.44.67:47281)
0808 23:28:12.735174 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:12.735223 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:12.736811 (+ 1588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:12.737583 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:12.737594 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:12.737705 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:12.738284 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:28:12.738287 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:12.738323 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:28:12.738405 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:28:12.738691 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:28:12.738696 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:28:12.738701 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:28:12.738896 (+ 195us) client_negotiation.cc:240] Negotiation successful
0808 23:28:12.739071 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":325,"threads_started":1}
I20260808 23:28:12.741083 25588 heartbeater.cc:347] Connected to a master server at 127.9.44.126:34517
I20260808 23:28:12.741338 25588 heartbeater.cc:464] Registering TS with master...
I20260808 23:28:12.741878 25588 heartbeater.cc:511] Master 127.9.44.126:34517 requested a full tablet report, sending...
I20260808 23:28:12.743125 24830 ts_manager.cc:194] Registered new tserver with Master: 445af285092a4996bee75ae1cd6010ec (127.9.44.67:45265)
I20260808 23:28:12.748123 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 23:28:12.753760 24899 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:28:12.752296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40674 (local address 127.9.44.126:34517)
0808 23:28:12.752429 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:12.752433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:12.752926 (+ 493us) server_negotiation.cc:407] Connection header received
0808 23:28:12.753080 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:12.753083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:12.753146 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:12.753223 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:12.753569 (+ 346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40674: BlockingRecv error: recv got EOF from 127.0.0.1:40674 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 23:28:12.754549 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 24942
I20260808 23:28:12.757697 25605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:12.755482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35036 (local address 127.9.44.65:36357)
0808 23:28:12.755991 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:12.755997 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:12.756021 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:28:12.756075 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:12.756080 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:12.756238 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:12.756348 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:12.757161 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:28:12.757166 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:28:12.757176 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:28:12.757195 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:28:12.757236 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:28:12.757250 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:28:12.757347 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:12.757414 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 23:28:12.757484 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":275,"threads_started":1}
I20260808 23:28:12.758760 25135 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:28:12.837318 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 24942
I20260808 23:28:12.855301 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 25148
I20260808 23:28:12.859000 25608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:12.856303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47944 (local address 127.9.44.66:44117)
0808 23:28:12.856828 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:12.856835 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:12.856870 (+ 35us) server_negotiation.cc:407] Connection header received
0808 23:28:12.856934 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:12.856940 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:12.857149 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:12.857294 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:12.858267 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:28:12.858273 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:28:12.858285 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:28:12.858299 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:28:12.858340 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:28:12.858354 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:28:12.858478 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:12.858629 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 23:28:12.858740 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":249,"threads_started":1}
I20260808 23:28:12.860293 25354 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:28:12.938993 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 25148
I20260808 23:28:12.964701 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 25376
I20260808 23:28:12.968406 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:12.965618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45804 (local address 127.9.44.67:45265)
0808 23:28:12.966316 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:12.966322 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:12.966352 (+ 30us) server_negotiation.cc:407] Connection header received
0808 23:28:12.966416 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:12.966422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:12.966637 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:12.966761 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:12.967637 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:28:12.967643 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:28:12.967655 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:28:12.967736 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:28:12.967784 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:28:12.967801 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:28:12.967929 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:12.967995 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 23:28:12.968086 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":349,"threads_started":1}
I20260808 23:28:12.969579 25577 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:28:13.048805 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 25376
I20260808 23:28:13.072466 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 24768
I20260808 23:28:13.073616 24882 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:28:13.180832 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 24768
2026-08-08T23:28:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2290 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:28:13 dist-test-slave-6bbx krb5kdc[25625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:28:13 dist-test-slave-6bbx krb5kdc[25625](info): set up 2 sockets
Aug 08 23:28:13 dist-test-slave-6bbx krb5kdc[25625](info): commencing operation
krb5kdc: starting...
W20260808 23:28:14.379351 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:28:14 dist-test-slave-6bbx krb5kdc[25625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:28:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:28:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:28:14.605203 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:41433
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:46801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:41433
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:28:14.919126 25646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:28:14.919391 25646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:28:14.919521 25646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:28:14.926642 25646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:28:14.926740 25646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:28:14.926817 25646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:28:14.926895 25646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:28:14.926954 25646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:28:14.936269 25646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:46801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:41433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:41433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:28:14.937448 25646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:28:14.938673 25646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:28:14.949487 25662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:28:14.949527 25653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:28:14.949561 25656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:28:14.950603 25646 server_base.cc:1034] running on GCE node
I20260808 23:28:14.951328 25646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:28:14.952666 25646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:28:14.953872 25646 hybrid_clock.cc:648] HybridClock initialized: now 1786231694953827 us; error 30 us; skew 500 ppm
I20260808 23:28:14.954200 25646 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:28:14 dist-test-slave-6bbx krb5kdc[25625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231694, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:28:14.957228 25646 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:28:14.958516 25646 webserver.cc:460] Webserver started at http://127.9.44.126:45051/ using document root <none> and password file <none>
I20260808 23:28:14.959143 25646 fs_manager.cc:359] Metadata directory not provided
I20260808 23:28:14.959254 25646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:28:14.959547 25646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:28:14.962345 25646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "4a201415a0ff42d2ae9cb87c9621a322"
format_stamp: "Formatted at 2026-08-08 23:28:14 on dist-test-slave-6bbx"
server_key: "3830c5963a42a9e6d1d2ec72997b4b4a"
server_key_iv: "5c672d323f2d5cbbb40df91741ecfed5"
server_key_version: "encryptionkey@0"
I20260808 23:28:14.963114 25646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "4a201415a0ff42d2ae9cb87c9621a322"
format_stamp: "Formatted at 2026-08-08 23:28:14 on dist-test-slave-6bbx"
server_key: "3830c5963a42a9e6d1d2ec72997b4b4a"
server_key_iv: "5c672d323f2d5cbbb40df91741ecfed5"
server_key_version: "encryptionkey@0"
I20260808 23:28:14.968055 25646 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 23:28:14.971192 25672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:14.972441 25646 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 23:28:14.972601 25646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "4a201415a0ff42d2ae9cb87c9621a322"
format_stamp: "Formatted at 2026-08-08 23:28:14 on dist-test-slave-6bbx"
server_key: "3830c5963a42a9e6d1d2ec72997b4b4a"
server_key_iv: "5c672d323f2d5cbbb40df91741ecfed5"
server_key_version: "encryptionkey@0"
I20260808 23:28:14.972810 25646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:28:14.987543 25646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:28:14.991295 25646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:28:14.991518 25646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:28:15.007735 25646 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:41433
I20260808 23:28:15.007756 25757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:41433 every 8 connection(s)
I20260808 23:28:15.008977 25646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:28:15.012404 25758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:28:15.013512 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 25646
I20260808 23:28:15.013708 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:28:15.014321 9393 external_mini_cluster.cc:1428] Setting key 121aefbc106883ccfbf8c658b3516160
I20260808 23:28:15.025658 25758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a201415a0ff42d2ae9cb87c9621a322: Bootstrap starting.
Aug 08 23:28:15 dist-test-slave-6bbx krb5kdc[25625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:28:15.028740 25758 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a201415a0ff42d2ae9cb87c9621a322: Neither blocks nor log segments found. Creating new log.
I20260808 23:28:15.029846 25758 log.cc:824] T 00000000000000000000000000000000 P 4a201415a0ff42d2ae9cb87c9621a322: Log is configured to *not* fsync() on all Append() calls
I20260808 23:28:15.032637 25758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a201415a0ff42d2ae9cb87c9621a322: No bootstrap required, opened a new log
I20260808 23:28:15.036464 25763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:15.016179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41562 (local address 127.9.44.126:41433)
0808 23:28:15.017020 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:15.017035 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:15.017088 (+ 53us) server_negotiation.cc:407] Connection header received
0808 23:28:15.018644 (+ 1556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:15.018678 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:15.019158 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:15.019581 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:15.020898 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:15.022875 (+ 1977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:15.024220 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:15.025185 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:15.025366 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:15.029431 (+ 4065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:28:15.029438 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:28:15.029452 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:28:15.029486 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:28:15.032534 (+ 3048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:28:15.033338 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:28:15.033344 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:28:15.033350 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:28:15.033517 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:28:15.033912 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:28:15.033915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:28:15.033917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:28:15.034460 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:28:15.034700 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:15.035121 (+ 421us) server_negotiation.cc:299] Negotiation successful
0808 23:28:15.035453 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":316,"threads_started":1}
I20260808 23:28:15.041082 25758 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a201415a0ff42d2ae9cb87c9621a322 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a201415a0ff42d2ae9cb87c9621a322" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 41433 } }
I20260808 23:28:15.041399 25758 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a201415a0ff42d2ae9cb87c9621a322 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:28:15.041495 25758 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a201415a0ff42d2ae9cb87c9621a322 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a201415a0ff42d2ae9cb87c9621a322, State: Initialized, Role: FOLLOWER
I20260808 23:28:15.042001 25758 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a201415a0ff42d2ae9cb87c9621a322 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a201415a0ff42d2ae9cb87c9621a322" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 41433 } }
I20260808 23:28:15.042223 25758 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a201415a0ff42d2ae9cb87c9621a322 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:28:15.042351 25758 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a201415a0ff42d2ae9cb87c9621a322 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:28:15.042500 25758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a201415a0ff42d2ae9cb87c9621a322 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:28:15.044718 25758 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a201415a0ff42d2ae9cb87c9621a322 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a201415a0ff42d2ae9cb87c9621a322" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 41433 } }
I20260808 23:28:15.045178 25758 leader_election.cc:302] T 00000000000000000000000000000000 P 4a201415a0ff42d2ae9cb87c9621a322 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a201415a0ff42d2ae9cb87c9621a322; no voters:
I20260808 23:28:15.045698 25758 leader_election.cc:288] T 00000000000000000000000000000000 P 4a201415a0ff42d2ae9cb87c9621a322 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:28:15.045804 25765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a201415a0ff42d2ae9cb87c9621a322 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:28:15.046442 25765 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a201415a0ff42d2ae9cb87c9621a322 [term 1 LEADER]: Becoming Leader. State: Replica: 4a201415a0ff42d2ae9cb87c9621a322, State: Running, Role: LEADER
I20260808 23:28:15.046950 25765 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a201415a0ff42d2ae9cb87c9621a322 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a201415a0ff42d2ae9cb87c9621a322" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 41433 } }
I20260808 23:28:15.047516 25758 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a201415a0ff42d2ae9cb87c9621a322 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:28:15.049307 25768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a201415a0ff42d2ae9cb87c9621a322 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a201415a0ff42d2ae9cb87c9621a322. Latest consensus state: current_term: 1 leader_uuid: "4a201415a0ff42d2ae9cb87c9621a322" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a201415a0ff42d2ae9cb87c9621a322" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 41433 } } }
I20260808 23:28:15.049602 25768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a201415a0ff42d2ae9cb87c9621a322 [sys.catalog]: This master's current role is: LEADER
I20260808 23:28:15.050134 25767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a201415a0ff42d2ae9cb87c9621a322 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a201415a0ff42d2ae9cb87c9621a322" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a201415a0ff42d2ae9cb87c9621a322" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 41433 } } }
I20260808 23:28:15.050473 25767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a201415a0ff42d2ae9cb87c9621a322 [sys.catalog]: This master's current role is: LEADER
I20260808 23:28:15.050493 25772 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:28:15.057003 25772 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:28:15.066565 25772 catalog_manager.cc:1347] Generated new cluster ID: da79f98bf5304de9b4aed28534ce07d3
I20260808 23:28:15.066663 25772 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:28:15.077920 25772 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:28:15.079677 25772 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:28:15.090349 25772 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a201415a0ff42d2ae9cb87c9621a322: Generated new TSK 0
I20260808 23:28:15.091202 25772 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.44.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
I20260808 23:28:15.163000 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:0
--local_ip_for_outbound_sockets=127.9.44.65
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:41433
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:46801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:28:15.515242 25808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:28:15.515493 25808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:28:15.515590 25808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:28:15.522871 25808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:28:15.522990 25808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:28:15.523138 25808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:28:15.532336 25808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:46801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:41433
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:28:15.533402 25808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:28:15.534584 25808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:28:15.546752 25816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:28:15.546773 25822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:28:15.546869 25819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:28:15.548098 25808 server_base.cc:1034] running on GCE node
I20260808 23:28:15.548717 25808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:28:15.549598 25808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:28:15.550786 25808 hybrid_clock.cc:648] HybridClock initialized: now 1786231695550742 us; error 36 us; skew 500 ppm
I20260808 23:28:15.551067 25808 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:28:15 dist-test-slave-6bbx krb5kdc[25625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231695, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:28:15.553969 25808 init.cc:377] Logged in from keytab as kudu/127.9.44.65@KRBTEST.COM (short username kudu)
I20260808 23:28:15.555279 25808 webserver.cc:460] Webserver started at http://127.9.44.65:37645/ using document root <none> and password file <none>
I20260808 23:28:15.555965 25808 fs_manager.cc:359] Metadata directory not provided
I20260808 23:28:15.556075 25808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:28:15.556365 25808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:28:15.559278 25808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/data/instance:
uuid: "7abcb1b9f33349068bcd1156bc4fd610"
format_stamp: "Formatted at 2026-08-08 23:28:15 on dist-test-slave-6bbx"
server_key: "2c682a8253d25eb91fe837e888f8ce31"
server_key_iv: "a1769861121a0001ef9490d36401a40e"
server_key_version: "encryptionkey@0"
I20260808 23:28:15.560036 25808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance:
uuid: "7abcb1b9f33349068bcd1156bc4fd610"
format_stamp: "Formatted at 2026-08-08 23:28:15 on dist-test-slave-6bbx"
server_key: "2c682a8253d25eb91fe837e888f8ce31"
server_key_iv: "a1769861121a0001ef9490d36401a40e"
server_key_version: "encryptionkey@0"
I20260808 23:28:15.564793 25808 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 23:28:15.567934 25835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:15.569128 25808 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 23:28:15.569278 25808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "7abcb1b9f33349068bcd1156bc4fd610"
format_stamp: "Formatted at 2026-08-08 23:28:15 on dist-test-slave-6bbx"
server_key: "2c682a8253d25eb91fe837e888f8ce31"
server_key_iv: "a1769861121a0001ef9490d36401a40e"
server_key_version: "encryptionkey@0"
I20260808 23:28:15.569449 25808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:28:15.601986 25808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:28:15.605922 25808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:28:15.606163 25808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:28:15.607007 25808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:28:15.608301 25808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:28:15.608371 25808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:15.608482 25808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:28:15.608530 25808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:15.638326 25808 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:33807
I20260808 23:28:15.638392 26031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:33807 every 8 connection(s)
I20260808 23:28:15.639477 25808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:28:15.648278 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 25808
I20260808 23:28:15.648506 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance
I20260808 23:28:15.649165 9393 external_mini_cluster.cc:1428] Setting key 064200a879f8749335c21dc2a2d2e41b
Aug 08 23:28:15 dist-test-slave-6bbx krb5kdc[25625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231695, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:28:15.658290 26037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:15.641826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:59603 (local address 127.9.44.126:41433)
0808 23:28:15.642239 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:15.642244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:15.643169 (+ 925us) server_negotiation.cc:407] Connection header received
0808 23:28:15.644994 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:15.645000 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:15.645083 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:15.645181 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:15.646602 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:15.647373 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:15.649049 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:15.649492 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:15.649606 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:15.653033 (+ 3427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:28:15.653038 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:28:15.653041 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:28:15.653073 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:28:15.655361 (+ 2288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:28:15.656078 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:28:15.656082 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:28:15.656084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:28:15.656205 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:28:15.656620 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:28:15.656624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:28:15.656626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:28:15.656890 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:28:15.657023 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:15.657684 (+ 661us) server_negotiation.cc:299] Negotiation successful
0808 23:28:15.657923 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":192,"threads_started":1}
I20260808 23:28:15.659067 26038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:15.642332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41433 (local address 127.9.44.65:59603)
0808 23:28:15.643016 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:15.643060 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:15.644724 (+ 1664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:15.645424 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:15.645433 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:15.645914 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:15.646434 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:15.646452 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:15.647521 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:15.647525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:15.648902 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:15.648916 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:15.649718 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:15.649727 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:15.649954 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:15.650576 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:15.650612 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:15.652819 (+ 2207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:28:15.655491 (+ 2672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:28:15.655499 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:28:15.655514 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:28:15.655954 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:28:15.656308 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:28:15.656314 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:28:15.656317 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:28:15.656504 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:28:15.657014 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:28:15.657021 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:28:15.657303 (+ 282us) client_negotiation.cc:769] Sending connection context
0808 23:28:15.657538 (+ 235us) client_negotiation.cc:240] Negotiation successful
0808 23:28:15.658005 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":270,"threads_started":1}
I20260808 23:28:15.660418 26035 heartbeater.cc:347] Connected to a master server at 127.9.44.126:41433
I20260808 23:28:15.660674 26035 heartbeater.cc:464] Registering TS with master...
I20260808 23:28:15.661336 26035 heartbeater.cc:511] Master 127.9.44.126:41433 requested a full tablet report, sending...
I20260808 23:28:15.663475 25706 ts_manager.cc:194] Registered new tserver with Master: 7abcb1b9f33349068bcd1156bc4fd610 (127.9.44.65:33807)
I20260808 23:28:15.665395 25706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.65@KRBTEST.COM'} at 127.9.44.65:59603
WARNING: no policy specified for kudu/127.9.44.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
I20260808 23:28:15.702471 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:0
--local_ip_for_outbound_sockets=127.9.44.66
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:41433
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:46801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:28:16.023343 26042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:28:16.023591 26042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:28:16.023710 26042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:28:16.030616 26042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:28:16.030728 26042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:28:16.030861 26042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:28:16.039943 26042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:46801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:41433
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:28:16.041028 26042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:28:16.042256 26042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:28:16.054889 26056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:28:16.054972 26049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:28:16.054946 26052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:28:16.055935 26042 server_base.cc:1034] running on GCE node
I20260808 23:28:16.056488 26042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:28:16.057332 26042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:28:16.058523 26042 hybrid_clock.cc:648] HybridClock initialized: now 1786231696058471 us; error 41 us; skew 500 ppm
I20260808 23:28:16.058802 26042 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:28:16 dist-test-slave-6bbx krb5kdc[25625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231696, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:28:16.061817 26042 init.cc:377] Logged in from keytab as kudu/127.9.44.66@KRBTEST.COM (short username kudu)
I20260808 23:28:16.063115 26042 webserver.cc:460] Webserver started at http://127.9.44.66:45009/ using document root <none> and password file <none>
I20260808 23:28:16.063753 26042 fs_manager.cc:359] Metadata directory not provided
I20260808 23:28:16.063865 26042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:28:16.064113 26042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:28:16.066795 26042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/data/instance:
uuid: "5a35ad4902654aa4bbf69ea0e94c13ad"
format_stamp: "Formatted at 2026-08-08 23:28:16 on dist-test-slave-6bbx"
server_key: "6eb674f935c993d92a4c69ca3d20484a"
server_key_iv: "fddfa8b5a973fa4b3447300ce327ba8b"
server_key_version: "encryptionkey@0"
I20260808 23:28:16.067610 26042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance:
uuid: "5a35ad4902654aa4bbf69ea0e94c13ad"
format_stamp: "Formatted at 2026-08-08 23:28:16 on dist-test-slave-6bbx"
server_key: "6eb674f935c993d92a4c69ca3d20484a"
server_key_iv: "fddfa8b5a973fa4b3447300ce327ba8b"
server_key_version: "encryptionkey@0"
I20260808 23:28:16.072540 26042 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260808 23:28:16.075881 26067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:16.077174 26042 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 23:28:16.077319 26042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "5a35ad4902654aa4bbf69ea0e94c13ad"
format_stamp: "Formatted at 2026-08-08 23:28:16 on dist-test-slave-6bbx"
server_key: "6eb674f935c993d92a4c69ca3d20484a"
server_key_iv: "fddfa8b5a973fa4b3447300ce327ba8b"
server_key_version: "encryptionkey@0"
I20260808 23:28:16.077490 26042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:28:16.099308 26042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:28:16.102835 26042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:28:16.103075 26042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:28:16.103982 26042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:28:16.105221 26042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:28:16.105293 26042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:16.105404 26042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:28:16.105463 26042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:16.134210 26042 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:38169
I20260808 23:28:16.134279 26242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:38169 every 8 connection(s)
I20260808 23:28:16.135404 26042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:28:16.142465 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 26042
I20260808 23:28:16.142680 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance
I20260808 23:28:16.143114 9393 external_mini_cluster.cc:1428] Setting key 449c5ed31fe3b9f3006643e0170a6260
Aug 08 23:28:16 dist-test-slave-6bbx krb5kdc[25625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231696, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:28:16.155522 26037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:16.137846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:58617 (local address 127.9.44.126:41433)
0808 23:28:16.138044 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:16.138050 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:16.139179 (+ 1129us) server_negotiation.cc:407] Connection header received
0808 23:28:16.140979 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:16.140983 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:16.141052 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:16.141136 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:16.142806 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.143544 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.145528 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.146144 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.146267 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:16.149655 (+ 3388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:28:16.149661 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:28:16.149665 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:28:16.149702 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:28:16.152701 (+ 2999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:28:16.153355 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:28:16.153367 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:28:16.153370 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:28:16.153533 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:28:16.153907 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:28:16.153911 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:28:16.153914 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:28:16.154226 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:28:16.154377 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:16.154977 (+ 600us) server_negotiation.cc:299] Negotiation successful
0808 23:28:16.155202 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 23:28:16.156266 26249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:16.138305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41433 (local address 127.9.44.66:58617)
0808 23:28:16.139022 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:16.139060 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:16.140726 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:16.141351 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:16.141359 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:16.141832 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:16.142598 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:16.142626 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.143730 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.143736 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:16.145388 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:16.145401 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.146319 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.146325 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:16.146614 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:16.147280 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:16.147302 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:16.149439 (+ 2137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:28:16.152827 (+ 3388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:28:16.152833 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:28:16.152844 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:28:16.153230 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:28:16.153633 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:28:16.153636 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:28:16.153638 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:28:16.153803 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:28:16.154352 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:28:16.154358 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:28:16.154640 (+ 282us) client_negotiation.cc:769] Sending connection context
0808 23:28:16.154848 (+ 208us) client_negotiation.cc:240] Negotiation successful
0808 23:28:16.155206 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":274,"threads_started":1}
I20260808 23:28:16.157603 26246 heartbeater.cc:347] Connected to a master server at 127.9.44.126:41433
I20260808 23:28:16.157939 26246 heartbeater.cc:464] Registering TS with master...
I20260808 23:28:16.158825 26246 heartbeater.cc:511] Master 127.9.44.126:41433 requested a full tablet report, sending...
I20260808 23:28:16.160306 25706 ts_manager.cc:194] Registered new tserver with Master: 5a35ad4902654aa4bbf69ea0e94c13ad (127.9.44.66:38169)
I20260808 23:28:16.161113 25706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.66@KRBTEST.COM'} at 127.9.44.66:58617
WARNING: no policy specified for kudu/127.9.44.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
I20260808 23:28:16.198388 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:0
--local_ip_for_outbound_sockets=127.9.44.67
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:41433
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:46801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:28:16.515571 26255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:28:16.515864 26255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:28:16.516001 26255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:28:16.522778 26255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:28:16.522886 26255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:28:16.523031 26255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:28:16.531967 26255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:46801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:41433
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:28:16.533061 26255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:28:16.534263 26255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:28:16.546350 26261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:28:16.546389 26265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:28:16.546314 26262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:28:16.547339 26255 server_base.cc:1034] running on GCE node
I20260808 23:28:16.547977 26255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:28:16.548854 26255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:28:16.550045 26255 hybrid_clock.cc:648] HybridClock initialized: now 1786231696549997 us; error 42 us; skew 500 ppm
I20260808 23:28:16.550325 26255 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:28:16 dist-test-slave-6bbx krb5kdc[25625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231696, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:28:16.553279 26255 init.cc:377] Logged in from keytab as kudu/127.9.44.67@KRBTEST.COM (short username kudu)
I20260808 23:28:16.554571 26255 webserver.cc:460] Webserver started at http://127.9.44.67:34515/ using document root <none> and password file <none>
I20260808 23:28:16.555177 26255 fs_manager.cc:359] Metadata directory not provided
I20260808 23:28:16.555289 26255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:28:16.555573 26255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:28:16.558383 26255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/data/instance:
uuid: "50fc6a806c9247529781bcec1732bef9"
format_stamp: "Formatted at 2026-08-08 23:28:16 on dist-test-slave-6bbx"
server_key: "c5ae32e65f582dfb79d8ba3f44d8ab54"
server_key_iv: "ac81d53efc0af873fefeefeac406558f"
server_key_version: "encryptionkey@0"
I20260808 23:28:16.559134 26255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance:
uuid: "50fc6a806c9247529781bcec1732bef9"
format_stamp: "Formatted at 2026-08-08 23:28:16 on dist-test-slave-6bbx"
server_key: "c5ae32e65f582dfb79d8ba3f44d8ab54"
server_key_iv: "ac81d53efc0af873fefeefeac406558f"
server_key_version: "encryptionkey@0"
I20260808 23:28:16.563997 26255 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260808 23:28:16.567130 26279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:16.568394 26255 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:28:16.568538 26255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "50fc6a806c9247529781bcec1732bef9"
format_stamp: "Formatted at 2026-08-08 23:28:16 on dist-test-slave-6bbx"
server_key: "c5ae32e65f582dfb79d8ba3f44d8ab54"
server_key_iv: "ac81d53efc0af873fefeefeac406558f"
server_key_version: "encryptionkey@0"
I20260808 23:28:16.568706 26255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:28:16.593179 26255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:28:16.596666 26255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:28:16.596915 26255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:28:16.597769 26255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:28:16.598995 26255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:28:16.599067 26255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:16.599179 26255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:28:16.599226 26255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:28:16.628984 26255 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:33719
I20260808 23:28:16.629034 26445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:33719 every 8 connection(s)
I20260808 23:28:16.630136 26255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:28:16.638635 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 26255
I20260808 23:28:16.638818 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance
I20260808 23:28:16.639369 9393 external_mini_cluster.cc:1428] Setting key ef8418cc757207d153f290156ef2817e
Aug 08 23:28:16 dist-test-slave-6bbx krb5kdc[25625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231696, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:28:16.650067 26037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:16.632510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:38543 (local address 127.9.44.126:41433)
0808 23:28:16.632699 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:16.632703 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:16.633735 (+ 1032us) server_negotiation.cc:407] Connection header received
0808 23:28:16.635437 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:16.635441 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:16.635510 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:16.635603 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:16.636979 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.637760 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.641024 (+ 3264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.641541 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.641673 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:16.644971 (+ 3298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:28:16.644976 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:28:16.644979 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:28:16.645010 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:28:16.647303 (+ 2293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:28:16.647995 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:28:16.647999 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:28:16.648001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:28:16.648121 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:28:16.648572 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:28:16.648575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:28:16.648577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:28:16.648841 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:28:16.648964 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:16.649541 (+ 577us) server_negotiation.cc:299] Negotiation successful
0808 23:28:16.649753 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 23:28:16.650733 26448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:16.632898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41433 (local address 127.9.44.67:38543)
0808 23:28:16.633565 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:16.633601 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:16.635186 (+ 1585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:16.635832 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:16.635839 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:16.636298 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:28:16.636824 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:16.636841 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.637919 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.637924 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:16.640829 (+ 2905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:16.640845 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.641759 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.641767 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:16.642029 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:16.642714 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:28:16.642735 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:28:16.644779 (+ 2044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:28:16.647427 (+ 2648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:28:16.647434 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:28:16.647443 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:28:16.647879 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:28:16.648241 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:28:16.648244 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:28:16.648247 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:28:16.648478 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:28:16.648948 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:28:16.648953 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:28:16.649244 (+ 291us) client_negotiation.cc:769] Sending connection context
0808 23:28:16.649450 (+ 206us) client_negotiation.cc:240] Negotiation successful
0808 23:28:16.649778 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":270,"threads_started":1}
I20260808 23:28:16.652024 26446 heartbeater.cc:347] Connected to a master server at 127.9.44.126:41433
I20260808 23:28:16.652308 26446 heartbeater.cc:464] Registering TS with master...
I20260808 23:28:16.652932 26446 heartbeater.cc:511] Master 127.9.44.126:41433 requested a full tablet report, sending...
I20260808 23:28:16.654381 25702 ts_manager.cc:194] Registered new tserver with Master: 50fc6a806c9247529781bcec1732bef9 (127.9.44.67:33719)
I20260808 23:28:16.655211 25702 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.67@KRBTEST.COM'} at 127.9.44.67:38543
I20260808 23:28:16.666079 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:28:16.668085 26035 heartbeater.cc:503] Master 127.9.44.126:41433 was elected leader, sending a full tablet report...
I20260808 23:28:16.681459 26037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:16.670444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41572 (local address 127.9.44.126:41433)
0808 23:28:16.670582 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:16.670586 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:16.671022 (+ 436us) server_negotiation.cc:407] Connection header received
0808 23:28:16.671147 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:16.671151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:16.671202 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:16.671273 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:16.672550 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.673359 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.674629 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.675021 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.675109 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:16.677155 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:28:16.677160 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:28:16.677163 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:28:16.677191 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:28:16.679305 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:28:16.679911 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:28:16.679914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:28:16.679916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:28:16.680025 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:28:16.680306 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:28:16.680309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:28:16.680311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:28:16.680610 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:28:16.680809 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:16.681070 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 23:28:16.681216 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 23:28:16.685046 25702 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41572:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:28:16.691576 25702 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:28:16.712497 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:16.702456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54990 (local address 127.9.44.66:38169)
0808 23:28:16.704669 (+ 2213us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:16.704679 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:16.704709 (+ 30us) server_negotiation.cc:407] Connection header received
0808 23:28:16.704813 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:16.704822 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:16.705082 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:16.706303 (+ 1221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:28:16.706521 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.707508 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.710233 (+ 2725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.711313 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.711452 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:16.711645 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:16.711990 (+ 345us) server_negotiation.cc:299] Negotiation successful
0808 23:28:16.712158 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2036,"thread_start_us":1878,"threads_started":1}
I20260808 23:28:16.712643 26466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:16.702339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:38169 (local address 127.0.0.1:54990)
0808 23:28:16.703853 (+ 1514us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:16.703919 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:16.704081 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:16.705620 (+ 1539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:16.705627 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:16.705658 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:28:16.706104 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:16.706118 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.707649 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.707654 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:16.710099 (+ 2445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:16.710110 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.711509 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.711515 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:16.711737 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:16.711798 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 23:28:16.711933 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 23:28:16.712223 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1264,"mutex_wait_us":76,"thread_start_us":2304,"threads_started":1}
I20260808 23:28:16.713332 26464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:16.702820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:33807 (local address 127.0.0.1:54236)
0808 23:28:16.704907 (+ 2087us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:16.704937 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:16.705044 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:16.706167 (+ 1123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:16.706172 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:16.706198 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:28:16.706607 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:16.706618 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.708773 (+ 2155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.708778 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:16.710782 (+ 2004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:16.710797 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.712590 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.712598 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:16.712769 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:16.712802 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 23:28:16.712875 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 23:28:16.712994 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1953,"thread_start_us":782,"threads_started":1}
I20260808 23:28:16.714495 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:16.703424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54236 (local address 127.9.44.65:33807)
0808 23:28:16.705576 (+ 2152us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:16.705584 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:16.705611 (+ 27us) server_negotiation.cc:407] Connection header received
0808 23:28:16.705718 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:16.705726 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:16.705948 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:16.706814 (+ 866us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:28:16.707045 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.708655 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.710950 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.712316 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.713750 (+ 1434us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:16.713964 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:16.714031 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 23:28:16.714184 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1989,"thread_start_us":1842,"threads_started":1}
I20260808 23:28:16.716418 26137 tablet_service.cc:1467] Processing CreateTablet for tablet 7459312ced834b208e92ad1b3951bd72 (DEFAULT_TABLE table=test-table [id=2cee8d173c8b4a178c6ae497e505aca0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:28:16.717725 26470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:16.705521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:33719 (local address 127.0.0.1:56794)
0808 23:28:16.708092 (+ 2571us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:16.708115 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:16.708217 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:16.709517 (+ 1300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:16.709522 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:16.709547 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:28:16.709979 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:16.709991 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.711612 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.711617 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:16.713960 (+ 2343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:16.713972 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.715284 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.715290 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:16.717114 (+ 1824us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:16.717149 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 23:28:16.717242 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 23:28:16.717356 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2449,"mutex_wait_us":23,"thread_start_us":2287,"threads_started":1}
I20260808 23:28:16.717763 26137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7459312ced834b208e92ad1b3951bd72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:28:16.717345 25920 tablet_service.cc:1467] Processing CreateTablet for tablet 7459312ced834b208e92ad1b3951bd72 (DEFAULT_TABLE table=test-table [id=2cee8d173c8b4a178c6ae497e505aca0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:28:16.718271 26468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:16.705264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56794 (local address 127.9.44.67:33719)
0808 23:28:16.708741 (+ 3477us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:16.708749 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:16.708779 (+ 30us) server_negotiation.cc:407] Connection header received
0808 23:28:16.708881 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:16.708888 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:16.709092 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:16.709260 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:28:16.710174 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.711449 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.714098 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.715092 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.715214 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:16.715435 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:16.717325 (+ 1890us) server_negotiation.cc:299] Negotiation successful
0808 23:28:16.717481 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3318,"thread_start_us":3173,"threads_started":1}
I20260808 23:28:16.718775 25920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7459312ced834b208e92ad1b3951bd72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:28:16.721575 26345 tablet_service.cc:1467] Processing CreateTablet for tablet 7459312ced834b208e92ad1b3951bd72 (DEFAULT_TABLE table=test-table [id=2cee8d173c8b4a178c6ae497e505aca0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:28:16.722913 26345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7459312ced834b208e92ad1b3951bd72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:28:16.729355 26471 tablet_bootstrap.cc:492] T 7459312ced834b208e92ad1b3951bd72 P 7abcb1b9f33349068bcd1156bc4fd610: Bootstrap starting.
I20260808 23:28:16.732307 26472 tablet_bootstrap.cc:492] T 7459312ced834b208e92ad1b3951bd72 P 5a35ad4902654aa4bbf69ea0e94c13ad: Bootstrap starting.
I20260808 23:28:16.732999 26471 tablet_bootstrap.cc:654] T 7459312ced834b208e92ad1b3951bd72 P 7abcb1b9f33349068bcd1156bc4fd610: Neither blocks nor log segments found. Creating new log.
I20260808 23:28:16.734261 26471 log.cc:824] T 7459312ced834b208e92ad1b3951bd72 P 7abcb1b9f33349068bcd1156bc4fd610: Log is configured to *not* fsync() on all Append() calls
I20260808 23:28:16.735739 26472 tablet_bootstrap.cc:654] T 7459312ced834b208e92ad1b3951bd72 P 5a35ad4902654aa4bbf69ea0e94c13ad: Neither blocks nor log segments found. Creating new log.
I20260808 23:28:16.735913 26474 tablet_bootstrap.cc:492] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9: Bootstrap starting.
I20260808 23:28:16.736985 26472 log.cc:824] T 7459312ced834b208e92ad1b3951bd72 P 5a35ad4902654aa4bbf69ea0e94c13ad: Log is configured to *not* fsync() on all Append() calls
I20260808 23:28:16.737138 26471 tablet_bootstrap.cc:492] T 7459312ced834b208e92ad1b3951bd72 P 7abcb1b9f33349068bcd1156bc4fd610: No bootstrap required, opened a new log
I20260808 23:28:16.737378 26471 ts_tablet_manager.cc:1397] T 7459312ced834b208e92ad1b3951bd72 P 7abcb1b9f33349068bcd1156bc4fd610: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 23:28:16.739553 26474 tablet_bootstrap.cc:654] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9: Neither blocks nor log segments found. Creating new log.
I20260808 23:28:16.740883 26474 log.cc:824] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9: Log is configured to *not* fsync() on all Append() calls
I20260808 23:28:16.742658 26472 tablet_bootstrap.cc:492] T 7459312ced834b208e92ad1b3951bd72 P 5a35ad4902654aa4bbf69ea0e94c13ad: No bootstrap required, opened a new log
I20260808 23:28:16.742924 26472 ts_tablet_manager.cc:1397] T 7459312ced834b208e92ad1b3951bd72 P 5a35ad4902654aa4bbf69ea0e94c13ad: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.008s
I20260808 23:28:16.743263 26474 tablet_bootstrap.cc:492] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9: No bootstrap required, opened a new log
I20260808 23:28:16.743480 26474 ts_tablet_manager.cc:1397] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260808 23:28:16.746673 26471 raft_consensus.cc:348] T 7459312ced834b208e92ad1b3951bd72 P 7abcb1b9f33349068bcd1156bc4fd610 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a35ad4902654aa4bbf69ea0e94c13ad" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38169 } } peers { permanent_uuid: "50fc6a806c9247529781bcec1732bef9" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33719 } } peers { permanent_uuid: "7abcb1b9f33349068bcd1156bc4fd610" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 33807 } }
I20260808 23:28:16.747097 26471 raft_consensus.cc:374] T 7459312ced834b208e92ad1b3951bd72 P 7abcb1b9f33349068bcd1156bc4fd610 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:28:16.747229 26471 raft_consensus.cc:729] T 7459312ced834b208e92ad1b3951bd72 P 7abcb1b9f33349068bcd1156bc4fd610 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7abcb1b9f33349068bcd1156bc4fd610, State: Initialized, Role: FOLLOWER
I20260808 23:28:16.747892 26471 consensus_queue.cc:260] T 7459312ced834b208e92ad1b3951bd72 P 7abcb1b9f33349068bcd1156bc4fd610 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a35ad4902654aa4bbf69ea0e94c13ad" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38169 } } peers { permanent_uuid: "50fc6a806c9247529781bcec1732bef9" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33719 } } peers { permanent_uuid: "7abcb1b9f33349068bcd1156bc4fd610" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 33807 } }
I20260808 23:28:16.749583 26471 ts_tablet_manager.cc:1428] T 7459312ced834b208e92ad1b3951bd72 P 7abcb1b9f33349068bcd1156bc4fd610: Time spent starting tablet: real 0.012s user 0.005s sys 0.008s
I20260808 23:28:16.752784 26474 raft_consensus.cc:348] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a35ad4902654aa4bbf69ea0e94c13ad" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38169 } } peers { permanent_uuid: "50fc6a806c9247529781bcec1732bef9" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33719 } } peers { permanent_uuid: "7abcb1b9f33349068bcd1156bc4fd610" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 33807 } }
I20260808 23:28:16.752792 26472 raft_consensus.cc:348] T 7459312ced834b208e92ad1b3951bd72 P 5a35ad4902654aa4bbf69ea0e94c13ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a35ad4902654aa4bbf69ea0e94c13ad" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38169 } } peers { permanent_uuid: "50fc6a806c9247529781bcec1732bef9" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33719 } } peers { permanent_uuid: "7abcb1b9f33349068bcd1156bc4fd610" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 33807 } }
I20260808 23:28:16.753098 26474 raft_consensus.cc:374] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:28:16.753189 26474 raft_consensus.cc:729] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50fc6a806c9247529781bcec1732bef9, State: Initialized, Role: FOLLOWER
I20260808 23:28:16.753203 26472 raft_consensus.cc:374] T 7459312ced834b208e92ad1b3951bd72 P 5a35ad4902654aa4bbf69ea0e94c13ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:28:16.753293 26472 raft_consensus.cc:729] T 7459312ced834b208e92ad1b3951bd72 P 5a35ad4902654aa4bbf69ea0e94c13ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a35ad4902654aa4bbf69ea0e94c13ad, State: Initialized, Role: FOLLOWER
I20260808 23:28:16.753847 26472 consensus_queue.cc:260] T 7459312ced834b208e92ad1b3951bd72 P 5a35ad4902654aa4bbf69ea0e94c13ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a35ad4902654aa4bbf69ea0e94c13ad" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38169 } } peers { permanent_uuid: "50fc6a806c9247529781bcec1732bef9" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33719 } } peers { permanent_uuid: "7abcb1b9f33349068bcd1156bc4fd610" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 33807 } }
I20260808 23:28:16.753772 26474 consensus_queue.cc:260] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a35ad4902654aa4bbf69ea0e94c13ad" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38169 } } peers { permanent_uuid: "50fc6a806c9247529781bcec1732bef9" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33719 } } peers { permanent_uuid: "7abcb1b9f33349068bcd1156bc4fd610" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 33807 } }
I20260808 23:28:16.754885 26246 heartbeater.cc:503] Master 127.9.44.126:41433 was elected leader, sending a full tablet report...
I20260808 23:28:16.755323 26472 ts_tablet_manager.cc:1428] T 7459312ced834b208e92ad1b3951bd72 P 5a35ad4902654aa4bbf69ea0e94c13ad: Time spent starting tablet: real 0.012s user 0.011s sys 0.002s
I20260808 23:28:16.756083 26446 heartbeater.cc:503] Master 127.9.44.126:41433 was elected leader, sending a full tablet report...
I20260808 23:28:16.756701 26474 ts_tablet_manager.cc:1428] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9: Time spent starting tablet: real 0.013s user 0.011s sys 0.000s
I20260808 23:28:16.846312 26480 raft_consensus.cc:482] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:28:16.846639 26480 raft_consensus.cc:504] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a35ad4902654aa4bbf69ea0e94c13ad" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38169 } } peers { permanent_uuid: "50fc6a806c9247529781bcec1732bef9" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33719 } } peers { permanent_uuid: "7abcb1b9f33349068bcd1156bc4fd610" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 33807 } }
I20260808 23:28:16.848639 26480 leader_election.cc:288] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a35ad4902654aa4bbf69ea0e94c13ad (127.9.44.66:38169), 7abcb1b9f33349068bcd1156bc4fd610 (127.9.44.65:33807)
I20260808 23:28:16.855043 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:16.849108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:50563 (local address 127.9.44.66:38169)
0808 23:28:16.849498 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:16.849505 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:16.849528 (+ 23us) server_negotiation.cc:407] Connection header received
0808 23:28:16.849674 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:16.849679 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:16.849764 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:16.849864 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:28:16.850314 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.851198 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.853100 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.854291 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.854415 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:16.854488 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:16.854764 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 23:28:16.854852 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222}
I20260808 23:28:16.855150 26448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:16.849085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:38169 (local address 127.9.44.67:50563)
0808 23:28:16.849340 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:16.849366 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:16.849500 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:16.849901 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:16.849909 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:16.849938 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:28:16.850141 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:16.850154 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.851329 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.851335 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:16.852943 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:16.852956 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.854432 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.854438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:16.854621 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:16.854655 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 23:28:16.854753 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 23:28:16.854884 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260808 23:28:16.856096 26173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7459312ced834b208e92ad1b3951bd72" candidate_uuid: "50fc6a806c9247529781bcec1732bef9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a35ad4902654aa4bbf69ea0e94c13ad" is_pre_election: true
I20260808 23:28:16.856356 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:16.849348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:47461 (local address 127.9.44.65:33807)
0808 23:28:16.849556 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:16.849562 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:16.850139 (+ 577us) server_negotiation.cc:407] Connection header received
0808 23:28:16.850351 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:16.850357 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:16.850426 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:16.850503 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:28:16.851075 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.852126 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.854376 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.855510 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.855624 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:16.855753 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:16.856003 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 23:28:16.856127 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260808 23:28:16.856436 26481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:16.849106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:33807 (local address 127.9.44.67:47461)
0808 23:28:16.850039 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0808 23:28:16.850053 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 23:28:16.850172 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:28:16.850541 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:28:16.850546 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:28:16.850569 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:28:16.850927 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:16.850940 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.852278 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.852284 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:16.854208 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:28:16.854225 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.855709 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.855716 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:28:16.855859 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:16.855882 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 23:28:16.855961 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 23:28:16.856075 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":803,"mutex_wait_us":4,"thread_start_us":575,"threads_started":1}
I20260808 23:28:16.856623 26173 raft_consensus.cc:2454] T 7459312ced834b208e92ad1b3951bd72 P 5a35ad4902654aa4bbf69ea0e94c13ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50fc6a806c9247529781bcec1732bef9 in term 0.
I20260808 23:28:16.857300 26287 leader_election.cc:302] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50fc6a806c9247529781bcec1732bef9, 5a35ad4902654aa4bbf69ea0e94c13ad; no voters:
I20260808 23:28:16.857324 25958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7459312ced834b208e92ad1b3951bd72" candidate_uuid: "50fc6a806c9247529781bcec1732bef9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7abcb1b9f33349068bcd1156bc4fd610" is_pre_election: true
I20260808 23:28:16.857663 26480 raft_consensus.cc:2781] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:28:16.857782 26480 raft_consensus.cc:482] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:28:16.857882 26480 raft_consensus.cc:3037] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:28:16.857836 25958 raft_consensus.cc:2454] T 7459312ced834b208e92ad1b3951bd72 P 7abcb1b9f33349068bcd1156bc4fd610 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50fc6a806c9247529781bcec1732bef9 in term 0.
I20260808 23:28:16.860301 26480 raft_consensus.cc:504] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a35ad4902654aa4bbf69ea0e94c13ad" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38169 } } peers { permanent_uuid: "50fc6a806c9247529781bcec1732bef9" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33719 } } peers { permanent_uuid: "7abcb1b9f33349068bcd1156bc4fd610" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 33807 } }
I20260808 23:28:16.860899 26480 leader_election.cc:288] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9 [CANDIDATE]: Term 1 election: Requested vote from peers 5a35ad4902654aa4bbf69ea0e94c13ad (127.9.44.66:38169), 7abcb1b9f33349068bcd1156bc4fd610 (127.9.44.65:33807)
I20260808 23:28:16.861274 26173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7459312ced834b208e92ad1b3951bd72" candidate_uuid: "50fc6a806c9247529781bcec1732bef9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a35ad4902654aa4bbf69ea0e94c13ad"
I20260808 23:28:16.861382 25958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7459312ced834b208e92ad1b3951bd72" candidate_uuid: "50fc6a806c9247529781bcec1732bef9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7abcb1b9f33349068bcd1156bc4fd610"
I20260808 23:28:16.861459 26173 raft_consensus.cc:3037] T 7459312ced834b208e92ad1b3951bd72 P 5a35ad4902654aa4bbf69ea0e94c13ad [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:28:16.861569 25958 raft_consensus.cc:3037] T 7459312ced834b208e92ad1b3951bd72 P 7abcb1b9f33349068bcd1156bc4fd610 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:28:16.863679 26173 raft_consensus.cc:2454] T 7459312ced834b208e92ad1b3951bd72 P 5a35ad4902654aa4bbf69ea0e94c13ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50fc6a806c9247529781bcec1732bef9 in term 1.
I20260808 23:28:16.863739 25958 raft_consensus.cc:2454] T 7459312ced834b208e92ad1b3951bd72 P 7abcb1b9f33349068bcd1156bc4fd610 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50fc6a806c9247529781bcec1732bef9 in term 1.
I20260808 23:28:16.864086 26287 leader_election.cc:302] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50fc6a806c9247529781bcec1732bef9, 5a35ad4902654aa4bbf69ea0e94c13ad; no voters:
I20260808 23:28:16.864368 26480 raft_consensus.cc:2781] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:28:16.864801 26480 raft_consensus.cc:686] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9 [term 1 LEADER]: Becoming Leader. State: Replica: 50fc6a806c9247529781bcec1732bef9, State: Running, Role: LEADER
I20260808 23:28:16.865257 26480 consensus_queue.cc:237] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a35ad4902654aa4bbf69ea0e94c13ad" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38169 } } peers { permanent_uuid: "50fc6a806c9247529781bcec1732bef9" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33719 } } peers { permanent_uuid: "7abcb1b9f33349068bcd1156bc4fd610" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 33807 } }
I20260808 23:28:16.869743 25702 catalog_manager.cc:5495] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50fc6a806c9247529781bcec1732bef9 (127.9.44.67). New cstate: current_term: 1 leader_uuid: "50fc6a806c9247529781bcec1732bef9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a35ad4902654aa4bbf69ea0e94c13ad" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50fc6a806c9247529781bcec1732bef9" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 33719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7abcb1b9f33349068bcd1156bc4fd610" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 33807 } health_report { overall_health: UNKNOWN } } }
W20260808 23:28:16.881486 26447 tablet.cc:2369] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:28:16.887143 26247 tablet.cc:2369] T 7459312ced834b208e92ad1b3951bd72 P 5a35ad4902654aa4bbf69ea0e94c13ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:28:16.891628 26036 tablet.cc:2369] T 7459312ced834b208e92ad1b3951bd72 P 7abcb1b9f33349068bcd1156bc4fd610: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:28:16.899250 26468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:16.894624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56798 (local address 127.9.44.67:33719)
0808 23:28:16.894787 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:16.894792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:16.894933 (+ 141us) server_negotiation.cc:407] Connection header received
0808 23:28:16.895109 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:16.895113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:16.895186 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:16.895303 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:28:16.895592 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.896329 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.897545 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.897929 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.898012 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:16.898396 (+ 384us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:28:16.898847 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:28:16.898965 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:16.899003 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 23:28:16.899064 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 23:28:16.906728 25958 raft_consensus.cc:1261] T 7459312ced834b208e92ad1b3951bd72 P 7abcb1b9f33349068bcd1156bc4fd610 [term 1 FOLLOWER]: Refusing update from remote peer 50fc6a806c9247529781bcec1732bef9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:28:16.906739 26173 raft_consensus.cc:1261] T 7459312ced834b208e92ad1b3951bd72 P 5a35ad4902654aa4bbf69ea0e94c13ad [term 1 FOLLOWER]: Refusing update from remote peer 50fc6a806c9247529781bcec1732bef9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:28:16.907562 26482 consensus_queue.cc:1035] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7abcb1b9f33349068bcd1156bc4fd610" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 33807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:28:16.907929 26480 consensus_queue.cc:1035] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a35ad4902654aa4bbf69ea0e94c13ad" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:28:16.918646 26490 mvcc.cc:205] Tried to move back new op lower bound from 7316405030520774656 to 7316405030363734016. Current Snapshot: MvccSnapshot[applied={T|T < 7316405030520774656}]
I20260808 23:28:16.937103 25702 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:28:16.937386 25702 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:28:16.940941 25702 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4a201415a0ff42d2ae9cb87c9621a322: Sending DeleteTablet for 3 replicas of tablet 7459312ced834b208e92ad1b3951bd72
I20260808 23:28:16.942025 26137 tablet_service.cc:1514] Processing DeleteTablet for tablet 7459312ced834b208e92ad1b3951bd72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:28:16 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:54990
I20260808 23:28:16.942349 26345 tablet_service.cc:1514] Processing DeleteTablet for tablet 7459312ced834b208e92ad1b3951bd72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:28:16 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:56794
I20260808 23:28:16.942898 25920 tablet_service.cc:1514] Processing DeleteTablet for tablet 7459312ced834b208e92ad1b3951bd72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:28:16 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:54236
I20260808 23:28:16.944840 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 25808
I20260808 23:28:16.949723 26502 tablet_replica.cc:330] T 7459312ced834b208e92ad1b3951bd72 P 5a35ad4902654aa4bbf69ea0e94c13ad: stopping tablet replica
I20260808 23:28:16.949977 26503 tablet_replica.cc:330] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9: stopping tablet replica
I20260808 23:28:16.950129 26502 raft_consensus.cc:2229] T 7459312ced834b208e92ad1b3951bd72 P 5a35ad4902654aa4bbf69ea0e94c13ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:28:16.950541 26503 raft_consensus.cc:2229] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:28:16.950616 26502 raft_consensus.cc:2258] T 7459312ced834b208e92ad1b3951bd72 P 5a35ad4902654aa4bbf69ea0e94c13ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:28:16.951268 26503 raft_consensus.cc:2258] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:28:16.953348 26503 ts_tablet_manager.cc:1905] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:28:16.954104 26502 ts_tablet_manager.cc:1905] T 7459312ced834b208e92ad1b3951bd72 P 5a35ad4902654aa4bbf69ea0e94c13ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:28:16.956167 26506 tablet_replica.cc:330] T 7459312ced834b208e92ad1b3951bd72 P 7abcb1b9f33349068bcd1156bc4fd610: stopping tablet replica
I20260808 23:28:16.956558 26506 raft_consensus.cc:2229] T 7459312ced834b208e92ad1b3951bd72 P 7abcb1b9f33349068bcd1156bc4fd610 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:28:16.957015 26506 raft_consensus.cc:2258] T 7459312ced834b208e92ad1b3951bd72 P 7abcb1b9f33349068bcd1156bc4fd610 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:28:16.960402 26503 ts_tablet_manager.cc:1918] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:28:16.960671 26503 log.cc:1196] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/7459312ced834b208e92ad1b3951bd72
I20260808 23:28:16.960873 26502 ts_tablet_manager.cc:1918] T 7459312ced834b208e92ad1b3951bd72 P 5a35ad4902654aa4bbf69ea0e94c13ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:28:16.961112 26502 log.cc:1196] T 7459312ced834b208e92ad1b3951bd72 P 5a35ad4902654aa4bbf69ea0e94c13ad: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/7459312ced834b208e92ad1b3951bd72
I20260808 23:28:16.961341 26503 ts_tablet_manager.cc:1939] T 7459312ced834b208e92ad1b3951bd72 P 50fc6a806c9247529781bcec1732bef9: Deleting consensus metadata
I20260808 23:28:16.961689 26502 ts_tablet_manager.cc:1939] T 7459312ced834b208e92ad1b3951bd72 P 5a35ad4902654aa4bbf69ea0e94c13ad: Deleting consensus metadata
I20260808 23:28:16.963124 25677 catalog_manager.cc:4841] TS 50fc6a806c9247529781bcec1732bef9 (127.9.44.67:33719): tablet 7459312ced834b208e92ad1b3951bd72 (table test-table [id=2cee8d173c8b4a178c6ae497e505aca0]) successfully deleted
I20260808 23:28:16.963552 26506 ts_tablet_manager.cc:1905] T 7459312ced834b208e92ad1b3951bd72 P 7abcb1b9f33349068bcd1156bc4fd610: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 23:28:16 dist-test-slave-6bbx krb5kdc[25625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.65@KRBTEST.COM
I20260808 23:28:16.963644 25677 catalog_manager.cc:4841] TS 5a35ad4902654aa4bbf69ea0e94c13ad (127.9.44.66:38169): tablet 7459312ced834b208e92ad1b3951bd72 (table test-table [id=2cee8d173c8b4a178c6ae497e505aca0]) successfully deleted
I20260808 23:28:16.970080 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:16.946992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54246 (local address 127.9.44.65:33807)
0808 23:28:16.947170 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:16.947177 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:16.953544 (+ 6367us) server_negotiation.cc:407] Connection header received
0808 23:28:16.953757 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:16.953763 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:16.953862 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:16.953969 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:16.955713 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.956939 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.958865 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:16.959472 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:16.959563 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:16.964588 (+ 5025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:28:16.964601 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:28:16.964613 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:28:16.964646 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:28:16.967324 (+ 2678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:28:16.968021 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:28:16.968029 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:28:16.968031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:28:16.968175 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:28:16.968544 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:28:16.968548 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:28:16.968550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:28:16.968923 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:28:16.969243 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:16.969489 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 23:28:16.969709 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 23:28:16.970607 26506 ts_tablet_manager.cc:1918] T 7459312ced834b208e92ad1b3951bd72 P 7abcb1b9f33349068bcd1156bc4fd610: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:28:16.970755 26025 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:28:16.970791 26506 log.cc:1196] T 7459312ced834b208e92ad1b3951bd72 P 7abcb1b9f33349068bcd1156bc4fd610: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786231597224934-9393-0/minicluster-data/ts-0/wal/wals/7459312ced834b208e92ad1b3951bd72
I20260808 23:28:17.059991 26506 ts_tablet_manager.cc:1939] T 7459312ced834b208e92ad1b3951bd72 P 7abcb1b9f33349068bcd1156bc4fd610: Deleting consensus metadata
I20260808 23:28:17.061730 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 25808
I20260808 23:28:17.061848 25676 catalog_manager.cc:4841] TS 7abcb1b9f33349068bcd1156bc4fd610 (127.9.44.65:33807): tablet 7459312ced834b208e92ad1b3951bd72 (table test-table [id=2cee8d173c8b4a178c6ae497e505aca0]) successfully deleted
I20260808 23:28:17.093436 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 26042
Aug 08 23:28:17 dist-test-slave-6bbx krb5kdc[25625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.66@KRBTEST.COM
I20260808 23:28:17.107920 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:17.094230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54994 (local address 127.9.44.66:38169)
0808 23:28:17.094438 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:17.094444 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:17.094462 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:28:17.094597 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:17.094601 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:17.094680 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:17.094776 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:17.096002 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:17.096728 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:17.097932 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:17.098325 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:17.098434 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:17.102568 (+ 4134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:28:17.102582 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:28:17.102599 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:28:17.102634 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:28:17.105315 (+ 2681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:28:17.106087 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:28:17.106093 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:28:17.106095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:28:17.106212 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:28:17.106565 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:28:17.106568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:28:17.106570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:28:17.106947 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:28:17.107153 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:17.107473 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 23:28:17.107626 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 23:28:17.108747 26236 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:28:17.204941 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 26042
I20260808 23:28:17.238085 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 26255
Aug 08 23:28:17 dist-test-slave-6bbx krb5kdc[25625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.67@KRBTEST.COM
I20260808 23:28:17.255887 26468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:28:17.238822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56814 (local address 127.9.44.67:33719)
0808 23:28:17.238999 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 23:28:17.239004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:28:17.239022 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:28:17.239241 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:28:17.239245 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:28:17.239309 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:28:17.239395 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:28:17.241230 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:17.242112 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:17.244115 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:28:17.244565 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:28:17.244684 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:28:17.250172 (+ 5488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:28:17.250184 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:28:17.250200 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:28:17.250243 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:28:17.253161 (+ 2918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:28:17.253830 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:28:17.253838 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:28:17.253840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:28:17.254012 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:28:17.254367 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:28:17.254371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:28:17.254373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:28:17.254728 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:28:17.255057 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0808 23:28:17.255323 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 23:28:17.255537 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 23:28:17.256608 26439 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:28:17.374235 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 26255
I20260808 23:28:17.396494 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 25646
I20260808 23:28:17.397590 25752 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:28:17.503353 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 25646
2026-08-08T23:28:17Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4328 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:28:17 dist-test-slave-6bbx krb5kdc[26523](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:28:17 dist-test-slave-6bbx krb5kdc[26523](info): set up 2 sockets
Aug 08 23:28:17 dist-test-slave-6bbx krb5kdc[26523](info): commencing operation
krb5kdc: starting...
W20260808 23:28:18.704779 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.001s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:28:18 dist-test-slave-6bbx krb5kdc[26523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:28:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:28:18Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.44.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.44.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.44.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/rangeradmin_127.9.44.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.44.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/rangeradmin_127.9.44.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.44.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.44.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.44.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/rangerlookup_127.9.44.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.44.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/rangerlookup_127.9.44.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.44.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.44.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.44.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/HTTP_127.9.44.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.44.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/HTTP_127.9.44.84@KRBTEST.COM.keytab.
I20260808 23:28:19.011551 9393 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskzx8VIc/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/postgres -l logfile start
2026-08-08 23:28:21.124 UTC [26563] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 23:28:21.124 UTC [26563] LOG: listening on IPv4 address "127.9.44.84", port 53633
2026-08-08 23:28:21.127 UTC [26563] LOG: listening on Unix socket "/tmp/.s.PGSQL.53633"
2026-08-08 23:28:21.147 UTC [26568] LOG: database system was shut down at 2026-08-08 23:28:20 UTC
2026-08-08 23:28:21.150 UTC [26563] LOG: database system is ready to accept connections
I20260808 23:28:22.220026 9393 mini_postgres.cc:96] Postgres bound to 53633
2026-08-08 23:28:22.244 UTC [26581] FATAL: database "slave" does not exist
127.9.44.84:53633 - accepting connections
I20260808 23:28:22.245744 9393 mini_ranger.cc:161] Starting Ranger...
I20260808 23:28:22.273137 9393 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 23:28:22.646294 9393 mini_ranger.cc:87] Created ranger Postgres database
I20260808 23:28:22.646406 9393 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ranger-admin
2026-08-08 23:28:22,981 [I] DB FLAVOR :POSTGRES
2026-08-08 23:28:22,981 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 23:28:22,981 [I] Checking connection..
2026-08-08 23:28:22,981 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 23:28:23,219 [I] Checking connection passed.
2026-08-08 23:28:23,219 [I] --------- Verifying version history table ---------
2026-08-08 23:28:23,219 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 23:28:23,475 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 23:28:23,475 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 23:28:23,725 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 23:28:23,725 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 23:28:23,725 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 23:28:23.971 UTC [26683] WARNING: there is no transaction in progress
2026-08-08 23:28:23,979 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 23:28:23,979 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 23:28:24,229 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 23:28:24,229 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 23:28:24,229 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 23:28:24,475 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 23:28:24,706 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6bbx.c.gcp-upstream.internal','N') ;"
2026-08-08 23:28:24,950 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 23:28:25,202 [I] Table x_portal_user does not exist in database ranger
2026-08-08 23:28:25,203 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 23:28:25,445 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 23:28:25,445 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 23:28:25,687 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 23:28:25,941 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 23:28:25,941 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 23:28:26.801 UTC [26848] WARNING: there is no transaction in progress
2026-08-08 23:28:26.815 UTC [26848] WARNING: there is no transaction in progress
2026-08-08 23:28:26.829 UTC [26848] WARNING: there is no transaction in progress
2026-08-08 23:28:26.842 UTC [26848] WARNING: there is no transaction in progress
2026-08-08 23:28:26.855 UTC [26848] WARNING: there is no transaction in progress
2026-08-08 23:28:26.869 UTC [26848] WARNING: there is no transaction in progress
2026-08-08 23:28:26.917 UTC [26848] WARNING: there is no transaction in progress
2026-08-08 23:28:26.923 UTC [26848] WARNING: there is no transaction in progress
2026-08-08 23:28:26.935 UTC [26848] WARNING: there is no transaction in progress
2026-08-08 23:28:26.944 UTC [26848] WARNING: there is no transaction in progress
2026-08-08 23:28:26.953 UTC [26848] WARNING: there is no transaction in progress
2026-08-08 23:28:26.965 UTC [26848] WARNING: there is no transaction in progress
2026-08-08 23:28:26.973 UTC [26848] WARNING: there is no transaction in progress
2026-08-08 23:28:26.978 UTC [26848] WARNING: there is no transaction in progress
2026-08-08 23:28:26.983 UTC [26848] WARNING: there is no transaction in progress
2026-08-08 23:28:26.988 UTC [26848] WARNING: there is no transaction in progress
2026-08-08 23:28:27,498 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 23:28:27,498 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 23:28:27,744 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 23:28:27,744 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 23:28:28,000 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 23:28:28,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 23:28:28,230 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 23:28:28,231 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 23:28:28,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 23:28:28,714 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6bbx.c.gcp-upstream.internal';"
2026-08-08 23:28:28,959 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 23:28:28,959 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:53633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 23:28:29,205 [I] DB_PATCHES have already been applied
I20260808 23:28:29.211338 9393 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ranger-admin:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskzx8VIc/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 23:28:29.211427 9393 mini_ranger.cc:193] Using host: 127.9.44.84
I20260808 23:28:29.231575 9393 mini_ranger.cc:227] Ranger admin URL: http://127.9.44.84:55905
Aug 08, 2026 11:28:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 11:28:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 11:28:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 11:28:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 11:28:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 11:28:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 11:28:30 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 23:28:30.453434 9393 mini_ranger.cc:160] Time spent starting Ranger: real 8.208s user 0.012s sys 0.044s
Aug 08, 2026 11:28:48 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 11:28:48 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 11:28:48 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 11:28:48 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 11:28:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 11:28:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 11:28:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 11:28:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 11:28:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 11:28:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 11:28:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 11:28:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 11:28:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 11:28:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 11:28:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 11:28:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 11:28:48 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 11:28:49 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 23:28:50.332706 9393 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.44.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.44.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.44.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/rangerkms_127.9.44.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.44.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/rangerkms_127.9.44.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.44.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.44.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.44.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/HTTP_127.9.44.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.44.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/HTTP_127.9.44.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 23:28:50 dist-test-slave-6bbx krb5kdc[26523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231730, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 23:28:50.506680 9393 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 23:28:50.527061 9393 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 23:28:51.198348 9393 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 23:28:51.198518 9393 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ranger-kms
I20260808 23:28:51.198647 9393 mini_ranger_kms.cc:227] Using postgres at 127.9.44.84:53633
2026-08-08 23:28:51,699 [I] DB FLAVOR :POSTGRES
2026-08-08 23:28:51,699 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 23:28:51,700 [I] Checking connection
2026-08-08 23:28:51,700 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:53633/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 23:28:51,977 [I] connection success
2026-08-08 23:28:51,977 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 23:28:51,977 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 23:28:51,977 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:53633/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 23:28:52,266 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 23:28:52,266 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 23:28:52,266 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 23:28:52,266 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:53633/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 23:28:52,548 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 23:28:52.985872 9393 mini_ranger_kms.cc:310] Created kms service
I20260808 23:28:53.192729 9393 mini_ranger_kms.cc:326] Created kudu user
I20260808 23:28:53.233460 9393 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 23:28:53 dist-test-slave-6bbx krb5kdc[26523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231730, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.44.84@KRBTEST.COM
I20260808 23:28:53.604189 9393 mini_ranger_kms.cc:384] Added ranger policy
I20260808 23:28:53.605563 9393 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ranger-kms:/tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 23:28:53.605646 9393 mini_ranger_kms.cc:240] Using host: 127.9.44.84
I20260808 23:28:53.632256 9393 mini_ranger_kms.cc:276] Ranger KMS PID: 27146
I20260808 23:28:53.632448 9393 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.44.84:55545
Aug 08, 2026 11:28:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 11:28:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 11:28:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 11:28:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 11:28:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 11:28:55 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 23:28:56 dist-test-slave-6bbx krb5kdc[26523](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786231736, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.44.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:28:56.195915 9393 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.689s user 0.126s sys 0.403s
I20260808 23:28:56.196250 9393 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 23:28:56.196487 9393 mini_ranger_kms.cc:401] 127.9.44.84:55545/kms/v1/keys
Aug 08 23:28:56 dist-test-slave-6bbx krb5kdc[26523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231736, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.44.84@KRBTEST.COM for HTTP/127.9.44.84@KRBTEST.COM
Aug 08 23:28:56 dist-test-slave-6bbx krb5kdc[26523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231736, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.44.84@KRBTEST.COM for HTTP/127.9.44.84@KRBTEST.COM
Aug 08 23:28:59 dist-test-slave-6bbx krb5kdc[26523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:28:59.296751 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:33453
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:39141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:33453
--ranger_config_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.44.84:55545/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:28:59.931905 27233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:28:59.932327 27233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:28:59.932554 27233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:28:59.943842 27233 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:28:59.943960 27233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:28:59.944039 27233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:28:59.944145 27233 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:28:59.944216 27233 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:28:59.956529 27233 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:39141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.44.84:55545/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:33453
--ranger_config_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:33453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:28:59.957878 27233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:28:59.959286 27233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:28:59.977587 27239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:28:59.977694 27242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:28:59.977770 27240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:28:59.978955 27233 server_base.cc:1034] running on GCE node
I20260808 23:28:59.979825 27233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:28:59.981365 27233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:28:59.982553 27233 hybrid_clock.cc:648] HybridClock initialized: now 1786231739982498 us; error 52 us; skew 500 ppm
I20260808 23:28:59.982863 27233 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:28:59 dist-test-slave-6bbx krb5kdc[26523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231739, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:28:59.988821 27233 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:28:59.990803 27233 webserver.cc:460] Webserver started at http://127.9.44.126:45885/ using document root <none> and password file <none>
I20260808 23:28:59.991489 27233 fs_manager.cc:359] Metadata directory not provided
I20260808 23:28:59.991703 27233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:28:59.992072 27233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 23:28:59 dist-test-slave-6bbx krb5kdc[26523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231739, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for HTTP/127.9.44.84@KRBTEST.COM
I20260808 23:29:00.046429 27233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "d5e04849307d482bba87bee9ca4cb68e"
format_stamp: "Formatted at 2026-08-08 23:29:00 on dist-test-slave-6bbx"
server_key: "ea8fb3df44fb7759ee23f7cd632efb3f"
server_key_iv: "04ae04a08f630184929dd0c2b0620a01"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:00.047489 27233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "d5e04849307d482bba87bee9ca4cb68e"
format_stamp: "Formatted at 2026-08-08 23:29:00 on dist-test-slave-6bbx"
server_key: "ea8fb3df44fb7759ee23f7cd632efb3f"
server_key_iv: "04ae04a08f630184929dd0c2b0620a01"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:00.055313 27233 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.005s
I20260808 23:29:00.074970 27252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:00.076637 27233 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260808 23:29:00.076881 27233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "d5e04849307d482bba87bee9ca4cb68e"
format_stamp: "Formatted at 2026-08-08 23:29:00 on dist-test-slave-6bbx"
server_key: "ea8fb3df44fb7759ee23f7cd632efb3f"
server_key_iv: "04ae04a08f630184929dd0c2b0620a01"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:00.077296 27233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:29:00.092762 27233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:29:00.098516 27233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:29:00.098958 27233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:29:00.156369 27233 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:33453
I20260808 23:29:00.156567 27343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:33453 every 8 connection(s)
I20260808 23:29:00.158231 27233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:29:00.164657 27344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:29:00.166033 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 27233
I20260808 23:29:00.166292 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:29:00.167352 9393 external_mini_cluster.cc:1428] Setting key c0a599f56ed15d73c409dde74904d115
I20260808 23:29:00.187498 27344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5e04849307d482bba87bee9ca4cb68e: Bootstrap starting.
I20260808 23:29:00.193696 27344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5e04849307d482bba87bee9ca4cb68e: Neither blocks nor log segments found. Creating new log.
I20260808 23:29:00.195490 27344 log.cc:824] T 00000000000000000000000000000000 P d5e04849307d482bba87bee9ca4cb68e: Log is configured to *not* fsync() on all Append() calls
I20260808 23:29:00.199321 27344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5e04849307d482bba87bee9ca4cb68e: No bootstrap required, opened a new log
Aug 08 23:29:00 dist-test-slave-6bbx krb5kdc[26523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:29:00.214166 27344 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5e04849307d482bba87bee9ca4cb68e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5e04849307d482bba87bee9ca4cb68e" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 33453 } }
I20260808 23:29:00.214892 27344 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5e04849307d482bba87bee9ca4cb68e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:29:00.215122 27344 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5e04849307d482bba87bee9ca4cb68e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5e04849307d482bba87bee9ca4cb68e, State: Initialized, Role: FOLLOWER
I20260808 23:29:00.215481 27349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:00.169948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33166 (local address 127.9.44.126:33453)
0808 23:29:00.183337 (+ 13389us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:00.183350 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:00.184467 (+ 1117us) server_negotiation.cc:407] Connection header received
0808 23:29:00.189030 (+ 4563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:00.189058 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:00.189624 (+ 566us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:00.190081 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:00.192411 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:00.194596 (+ 2185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:00.196567 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:00.197779 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:00.197979 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:00.203474 (+ 5495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:00.203489 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:00.203507 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:00.203551 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:00.209917 (+ 6366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:00.210746 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:00.210756 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:00.210763 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:00.211091 (+ 328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:00.211580 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:00.211585 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:00.211588 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:00.212483 (+ 895us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:00.212930 (+ 447us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:00.213330 (+ 400us) server_negotiation.cc:299] Negotiation successful
0808 23:29:00.213768 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13076,"thread_start_us":12658,"threads_started":1}
I20260808 23:29:00.216173 27344 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5e04849307d482bba87bee9ca4cb68e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5e04849307d482bba87bee9ca4cb68e" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 33453 } }
I20260808 23:29:00.216630 27344 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5e04849307d482bba87bee9ca4cb68e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:29:00.216899 27344 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5e04849307d482bba87bee9ca4cb68e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:29:00.217240 27344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5e04849307d482bba87bee9ca4cb68e [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:29:00.221333 27344 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5e04849307d482bba87bee9ca4cb68e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5e04849307d482bba87bee9ca4cb68e" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 33453 } }
I20260808 23:29:00.222010 27344 leader_election.cc:302] T 00000000000000000000000000000000 P d5e04849307d482bba87bee9ca4cb68e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5e04849307d482bba87bee9ca4cb68e; no voters:
I20260808 23:29:00.223280 27344 leader_election.cc:288] T 00000000000000000000000000000000 P d5e04849307d482bba87bee9ca4cb68e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:29:00.223583 27354 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5e04849307d482bba87bee9ca4cb68e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:29:00.226536 27354 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5e04849307d482bba87bee9ca4cb68e [term 1 LEADER]: Becoming Leader. State: Replica: d5e04849307d482bba87bee9ca4cb68e, State: Running, Role: LEADER
I20260808 23:29:00.227217 27354 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5e04849307d482bba87bee9ca4cb68e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5e04849307d482bba87bee9ca4cb68e" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 33453 } }
I20260808 23:29:00.228950 27344 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5e04849307d482bba87bee9ca4cb68e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:29:00.231230 27358 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5e04849307d482bba87bee9ca4cb68e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5e04849307d482bba87bee9ca4cb68e. Latest consensus state: current_term: 1 leader_uuid: "d5e04849307d482bba87bee9ca4cb68e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5e04849307d482bba87bee9ca4cb68e" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 33453 } } }
I20260808 23:29:00.232213 27358 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5e04849307d482bba87bee9ca4cb68e [sys.catalog]: This master's current role is: LEADER
I20260808 23:29:00.232165 27356 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5e04849307d482bba87bee9ca4cb68e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5e04849307d482bba87bee9ca4cb68e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5e04849307d482bba87bee9ca4cb68e" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 33453 } } }
I20260808 23:29:00.240247 27356 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5e04849307d482bba87bee9ca4cb68e [sys.catalog]: This master's current role is: LEADER
I20260808 23:29:00.243237 27361 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:29:00.255303 27361 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:29:00.259835 27344 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 23:29:00.283079 27361 catalog_manager.cc:1347] Generated new cluster ID: 68071e13d1af4d6eb6940102329e3c58
I20260808 23:29:00.283224 27361 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:29:00.315979 27361 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:29:00.318428 27361 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:29:00.328292 27361 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5e04849307d482bba87bee9ca4cb68e: Generated new TSK 0
I20260808 23:29:00.329375 27361 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.44.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
I20260808 23:29:01.247924 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.44.84:55545/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:0
--local_ip_for_outbound_sockets=127.9.44.65
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:33453
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:39141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:29:01 dist-test-slave-6bbx krb5kdc[26523](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786231741, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 23:29:01.936517 27412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:29:01.937023 27412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:29:01.937327 27412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:29:01.950275 27412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:29:01.950492 27412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:29:01.950757 27412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:29:01.966964 27412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:39141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.44.84:55545/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:33453
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:29:01.968473 27412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:29:01.970220 27412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:29:01.997897 27433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:29:02.000785 27435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:29:02.002362 27412 server_base.cc:1034] running on GCE node
W20260808 23:29:02.003044 27440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:29:02.003935 27412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:29:02.009240 27412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:29:02.010537 27412 hybrid_clock.cc:648] HybridClock initialized: now 1786231742010422 us; error 100 us; skew 500 ppm
I20260808 23:29:02.010929 27412 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:29:02 dist-test-slave-6bbx krb5kdc[26523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231742, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:29:02.015575 27412 init.cc:377] Logged in from keytab as kudu/127.9.44.65@KRBTEST.COM (short username kudu)
I20260808 23:29:02.018260 27412 webserver.cc:460] Webserver started at http://127.9.44.65:43713/ using document root <none> and password file <none>
I20260808 23:29:02.019498 27412 fs_manager.cc:359] Metadata directory not provided
I20260808 23:29:02.019801 27412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:29:02.020352 27412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 23:29:02 dist-test-slave-6bbx krb5kdc[26523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231742, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for HTTP/127.9.44.84@KRBTEST.COM
I20260808 23:29:02.048095 27412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-0/data/instance:
uuid: "dc5325b3607345aa9760768125671bc2"
format_stamp: "Formatted at 2026-08-08 23:29:02 on dist-test-slave-6bbx"
server_key: "a42b3df0198d92ba192ca1421e1692fe"
server_key_iv: "eac66b35561bde098f8e82159b7f7dec"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:02.049752 27412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance:
uuid: "dc5325b3607345aa9760768125671bc2"
format_stamp: "Formatted at 2026-08-08 23:29:02 on dist-test-slave-6bbx"
server_key: "a42b3df0198d92ba192ca1421e1692fe"
server_key_iv: "eac66b35561bde098f8e82159b7f7dec"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:02.056905 27412 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260808 23:29:02.088207 27455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:02.089895 27412 fs_manager.cc:725] Time spent opening block manager: real 0.018s user 0.009s sys 0.007s
I20260808 23:29:02.090133 27412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "dc5325b3607345aa9760768125671bc2"
format_stamp: "Formatted at 2026-08-08 23:29:02 on dist-test-slave-6bbx"
server_key: "a42b3df0198d92ba192ca1421e1692fe"
server_key_iv: "eac66b35561bde098f8e82159b7f7dec"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:02.090469 27412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 08 23:29:02 dist-test-slave-6bbx krb5kdc[26523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231741, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for HTTP/127.9.44.84@KRBTEST.COM
I20260808 23:29:02.145318 27412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:29:02.152099 27412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:29:02.152501 27412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:29:02.158777 27412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:29:02.163820 27412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:29:02.163921 27412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:02.164113 27412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:29:02.164179 27412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:02.358558 27412 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:37161
I20260808 23:29:02.358632 27640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:37161 every 8 connection(s)
I20260808 23:29:02.360739 27412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:29:02.367023 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 27412
I20260808 23:29:02.367249 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance
I20260808 23:29:02.368136 9393 external_mini_cluster.cc:1428] Setting key 8e0117da33a7b89033068b68343cb8d4
WARNING: no policy specified for kudu/127.9.44.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.66@KRBTEST.COM" created.
Aug 08 23:29:02 dist-test-slave-6bbx krb5kdc[26523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231742, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:29:02.450534 27645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:02.364009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:39695 (local address 127.9.44.126:33453)
0808 23:29:02.393583 (+ 29574us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:02.393590 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:02.393616 (+ 26us) server_negotiation.cc:407] Connection header received
0808 23:29:02.414701 (+ 21085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:02.414709 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:02.414812 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:02.414947 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:02.423803 (+ 8856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:02.424900 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:02.427989 (+ 3089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:02.428974 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:02.429071 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:02.440726 (+ 11655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:02.440734 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:02.440740 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:02.440782 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:02.445418 (+ 4636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:02.446761 (+ 1343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:02.446767 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:02.446771 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:02.447055 (+ 284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:02.447821 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:02.447826 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:02.447830 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:02.448337 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:02.448483 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:02.449698 (+ 1215us) server_negotiation.cc:299] Negotiation successful
0808 23:29:02.449978 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29386,"thread_start_us":29602,"threads_started":1}
I20260808 23:29:02.453495 27646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:02.365184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:33453 (local address 127.9.44.65:39695)
0808 23:29:02.387817 (+ 22633us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:02.387875 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:02.413011 (+ 25136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:02.415517 (+ 2506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:02.415531 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:02.416802 (+ 1271us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:29:02.423546 (+ 6744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:02.423575 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:02.425101 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:02.425106 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:02.427830 (+ 2724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:02.427845 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:02.429614 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:02.429622 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:02.429979 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:02.431033 (+ 1054us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:29:02.431069 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:29:02.440414 (+ 9345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:29:02.445657 (+ 5243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:02.445668 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:02.445686 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:02.446595 (+ 909us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:02.447226 (+ 631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:02.447231 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:02.447234 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:02.447657 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:02.448588 (+ 931us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:29:02.448596 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:29:02.449093 (+ 497us) client_negotiation.cc:769] Sending connection context
0808 23:29:02.451168 (+ 2075us) client_negotiation.cc:240] Negotiation successful
0808 23:29:02.451859 (+ 691us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22251,"thread_start_us":26374,"threads_started":1}
I20260808 23:29:02.455720 27641 heartbeater.cc:347] Connected to a master server at 127.9.44.126:33453
I20260808 23:29:02.456085 27641 heartbeater.cc:464] Registering TS with master...
I20260808 23:29:02.457662 27641 heartbeater.cc:511] Master 127.9.44.126:33453 requested a full tablet report, sending...
I20260808 23:29:02.461401 27278 ts_manager.cc:194] Registered new tserver with Master: dc5325b3607345aa9760768125671bc2 (127.9.44.65:37161)
I20260808 23:29:02.463907 27278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.65@KRBTEST.COM'} at 127.9.44.65:39695
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
I20260808 23:29:02.479034 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.44.84:55545/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:0
--local_ip_for_outbound_sockets=127.9.44.66
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:33453
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:39141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:29:02 dist-test-slave-6bbx krb5kdc[26523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231741, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for HTTP/127.9.44.84@KRBTEST.COM
W20260808 23:29:03.261452 27649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:29:03.261740 27649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:29:03.261878 27649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:29:03.269694 27649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:29:03.269928 27649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:29:03.270143 27649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:29:03.283069 27649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:39141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.44.84:55545/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:33453
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:29:03.284499 27649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:29:03.286062 27649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:29:03.304730 27678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:29:03.304805 27680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:29:03.307576 27677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:29:03.310527 27649 server_base.cc:1034] running on GCE node
I20260808 23:29:03.311504 27649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:29:03.312538 27649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:29:03.313859 27649 hybrid_clock.cc:648] HybridClock initialized: now 1786231743313722 us; error 43 us; skew 500 ppm
I20260808 23:29:03.314275 27649 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:29:03 dist-test-slave-6bbx krb5kdc[26523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231743, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:29:03.318140 27649 init.cc:377] Logged in from keytab as kudu/127.9.44.66@KRBTEST.COM (short username kudu)
I20260808 23:29:03.320065 27649 webserver.cc:460] Webserver started at http://127.9.44.66:34163/ using document root <none> and password file <none>
I20260808 23:29:03.321036 27649 fs_manager.cc:359] Metadata directory not provided
I20260808 23:29:03.321249 27649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:29:03.321692 27649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 23:29:03 dist-test-slave-6bbx krb5kdc[26523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231743, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for HTTP/127.9.44.84@KRBTEST.COM
I20260808 23:29:03.337505 27649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-1/data/instance:
uuid: "4d015d84d7d347d29b15304f2c44636c"
format_stamp: "Formatted at 2026-08-08 23:29:03 on dist-test-slave-6bbx"
server_key: "d1c836137c20454754c6840e3e3be409"
server_key_iv: "f31bcd4271299ed423cf81528f4436ab"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:03.338608 27649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance:
uuid: "4d015d84d7d347d29b15304f2c44636c"
format_stamp: "Formatted at 2026-08-08 23:29:03 on dist-test-slave-6bbx"
server_key: "d1c836137c20454754c6840e3e3be409"
server_key_iv: "f31bcd4271299ed423cf81528f4436ab"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:03.346097 27649 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260808 23:29:03.367111 27691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:03.369808 27649 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260808 23:29:03.370059 27649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "4d015d84d7d347d29b15304f2c44636c"
format_stamp: "Formatted at 2026-08-08 23:29:03 on dist-test-slave-6bbx"
server_key: "d1c836137c20454754c6840e3e3be409"
server_key_iv: "f31bcd4271299ed423cf81528f4436ab"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:03.370414 27649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:29:03.389647 27649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:29:03.395207 27649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:29:03.395817 27649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:29:03.397500 27649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:29:03.399953 27649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:29:03.400102 27649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:03.400470 27649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:29:03.400571 27649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:03.468092 27641 heartbeater.cc:503] Master 127.9.44.126:33453 was elected leader, sending a full tablet report...
I20260808 23:29:03.510275 27649 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:45213
I20260808 23:29:03.510448 27859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:45213 every 8 connection(s)
I20260808 23:29:03.512346 27649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:29:03.520612 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 27649
I20260808 23:29:03.520818 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance
I20260808 23:29:03.521467 9393 external_mini_cluster.cc:1428] Setting key fbe21c39560a6f6d7eecae241411ce23
Aug 08 23:29:03 dist-test-slave-6bbx krb5kdc[26523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231743, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:29:03.546314 27866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:03.516312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:38675 (local address 127.9.44.126:33453)
0808 23:29:03.517308 (+ 996us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:03.517313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:03.518937 (+ 1624us) server_negotiation.cc:407] Connection header received
0808 23:29:03.522212 (+ 3275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:03.522219 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:03.522308 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:03.522426 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:03.524807 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:03.526436 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:03.529328 (+ 2892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:03.530194 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:03.530318 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:03.537422 (+ 7104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:03.537428 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:03.537431 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:03.537496 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:03.541761 (+ 4265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:03.542798 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:03.542803 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:03.542806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:03.543049 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:03.543748 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:03.543753 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:03.543757 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:03.544273 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:03.544414 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:03.545376 (+ 962us) server_negotiation.cc:299] Negotiation successful
0808 23:29:03.545609 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":860,"thread_start_us":670,"threads_started":1}
I20260808 23:29:03.547539 27868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:03.517288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:33453 (local address 127.9.44.66:38675)
0808 23:29:03.518765 (+ 1477us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:03.518811 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:03.521865 (+ 3054us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:03.522751 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:03.522763 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:03.523471 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:29:03.524599 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:03.524636 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:03.526621 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:03.526632 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:03.529168 (+ 2536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:03.529184 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:03.530534 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:03.530544 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:03.530972 (+ 428us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:03.531875 (+ 903us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:29:03.531916 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:29:03.537136 (+ 5220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:29:03.541913 (+ 4777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:03.541922 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:03.541938 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:03.542650 (+ 712us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:03.543172 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:03.543177 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:03.543181 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:03.543585 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:03.544412 (+ 827us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:29:03.544420 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:29:03.544916 (+ 496us) client_negotiation.cc:769] Sending connection context
0808 23:29:03.545273 (+ 357us) client_negotiation.cc:240] Negotiation successful
0808 23:29:03.545860 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1184,"thread_start_us":908,"threads_started":1}
I20260808 23:29:03.549655 27861 heartbeater.cc:347] Connected to a master server at 127.9.44.126:33453
I20260808 23:29:03.550066 27861 heartbeater.cc:464] Registering TS with master...
I20260808 23:29:03.551440 27861 heartbeater.cc:511] Master 127.9.44.126:33453 requested a full tablet report, sending...
I20260808 23:29:03.553599 27278 ts_manager.cc:194] Registered new tserver with Master: 4d015d84d7d347d29b15304f2c44636c (127.9.44.66:45213)
I20260808 23:29:03.554857 27278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.66@KRBTEST.COM'} at 127.9.44.66:38675
WARNING: no policy specified for kudu/127.9.44.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
I20260808 23:29:03.592103 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.44.84:55545/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:0
--local_ip_for_outbound_sockets=127.9.44.67
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:33453
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:39141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:29:04.270916 27872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:29:04.271303 27872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:29:04.271566 27872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:29:04.282145 27872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:29:04.282356 27872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:29:04.282598 27872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:29:04.296217 27872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:39141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.44.84:55545/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:33453
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:29:04.297549 27872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:29:04.298930 27872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:29:04.317301 27879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:29:04.317483 27881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:29:04.318255 27883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:29:04.319150 27872 server_base.cc:1034] running on GCE node
I20260808 23:29:04.319949 27872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:29:04.321084 27872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:29:04.322352 27872 hybrid_clock.cc:648] HybridClock initialized: now 1786231744322198 us; error 76 us; skew 500 ppm
I20260808 23:29:04.322741 27872 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:29:04 dist-test-slave-6bbx krb5kdc[26523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231744, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:29:04.327235 27872 init.cc:377] Logged in from keytab as kudu/127.9.44.67@KRBTEST.COM (short username kudu)
I20260808 23:29:04.329190 27872 webserver.cc:460] Webserver started at http://127.9.44.67:43501/ using document root <none> and password file <none>
I20260808 23:29:04.330076 27872 fs_manager.cc:359] Metadata directory not provided
I20260808 23:29:04.330246 27872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:29:04.330631 27872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 23:29:04 dist-test-slave-6bbx krb5kdc[26523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231744, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for HTTP/127.9.44.84@KRBTEST.COM
I20260808 23:29:04.345986 27872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-2/data/instance:
uuid: "6fc86ab343dd40fd9ab697e60dab1a08"
format_stamp: "Formatted at 2026-08-08 23:29:04 on dist-test-slave-6bbx"
server_key: "408a23d922a9a4401c2e0ea4765fe93a"
server_key_iv: "b0297507997c6dee463573e68dca5356"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:04.346962 27872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance:
uuid: "6fc86ab343dd40fd9ab697e60dab1a08"
format_stamp: "Formatted at 2026-08-08 23:29:04 on dist-test-slave-6bbx"
server_key: "408a23d922a9a4401c2e0ea4765fe93a"
server_key_iv: "b0297507997c6dee463573e68dca5356"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:04.353628 27872 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.009s
I20260808 23:29:04.371294 27893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:04.372958 27872 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.005s
I20260808 23:29:04.373220 27872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "6fc86ab343dd40fd9ab697e60dab1a08"
format_stamp: "Formatted at 2026-08-08 23:29:04 on dist-test-slave-6bbx"
server_key: "408a23d922a9a4401c2e0ea4765fe93a"
server_key_iv: "b0297507997c6dee463573e68dca5356"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:04.373533 27872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:29:04.399006 27872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:29:04.404434 27872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:29:04.404970 27872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:29:04.406697 27872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:29:04.409116 27872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:29:04.409269 27872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:04.409489 27872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:29:04.409621 27872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:04.500211 27872 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:42923
I20260808 23:29:04.500252 28057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:42923 every 8 connection(s)
I20260808 23:29:04.501864 27872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:29:04.503211 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 27872
I20260808 23:29:04.503403 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance
I20260808 23:29:04.504261 9393 external_mini_cluster.cc:1428] Setting key 6aa009f308838e6a3604248e5c75c310
Aug 08 23:29:04 dist-test-slave-6bbx krb5kdc[26523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231744, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:29:04.530268 28062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:04.506113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:52091 (local address 127.9.44.126:33453)
0808 23:29:04.507216 (+ 1103us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:04.507222 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:04.509200 (+ 1978us) server_negotiation.cc:407] Connection header received
0808 23:29:04.512212 (+ 3012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:04.512219 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:04.512304 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:04.512401 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:04.514516 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.515548 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.517763 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.518348 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.518463 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:04.523115 (+ 4652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:04.523120 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:04.523142 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:04.523185 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:04.527039 (+ 3854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:04.527839 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:04.527843 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:04.527845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:04.528043 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:04.528546 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:04.528549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:04.528551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:04.528869 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:04.529043 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:04.529570 (+ 527us) server_negotiation.cc:299] Negotiation successful
0808 23:29:04.529728 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":968,"thread_start_us":707,"threads_started":1}
I20260808 23:29:04.530928 28063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:04.507143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:33453 (local address 127.9.44.67:52091)
0808 23:29:04.509062 (+ 1919us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:04.509099 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:04.511963 (+ 2864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:04.512679 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:04.512688 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:04.513420 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:29:04.514353 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:04.514373 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.515736 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.515740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:04.517618 (+ 1878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:04.517638 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.518702 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.518713 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:04.519039 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:04.519861 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:29:04.519911 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:29:04.522899 (+ 2988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:29:04.527171 (+ 4272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:04.527180 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:04.527191 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:04.527742 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:04.528123 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:04.528126 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:04.528128 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:04.528455 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:04.528962 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:29:04.528967 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:29:04.529265 (+ 298us) client_negotiation.cc:769] Sending connection context
0808 23:29:04.529508 (+ 243us) client_negotiation.cc:240] Negotiation successful
0808 23:29:04.529930 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1569,"thread_start_us":1214,"threads_started":1}
I20260808 23:29:04.532604 28058 heartbeater.cc:347] Connected to a master server at 127.9.44.126:33453
I20260808 23:29:04.532960 28058 heartbeater.cc:464] Registering TS with master...
I20260808 23:29:04.533926 28058 heartbeater.cc:511] Master 127.9.44.126:33453 requested a full tablet report, sending...
I20260808 23:29:04.536073 27276 ts_manager.cc:194] Registered new tserver with Master: 6fc86ab343dd40fd9ab697e60dab1a08 (127.9.44.67:42923)
I20260808 23:29:04.536772 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:29:04.537694 27276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.67@KRBTEST.COM'} at 127.9.44.67:52091
I20260808 23:29:04.558313 27861 heartbeater.cc:503] Master 127.9.44.126:33453 was elected leader, sending a full tablet report...
I20260808 23:29:04.560649 28062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:04.543403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33180 (local address 127.9.44.126:33453)
0808 23:29:04.543609 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:04.543615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:04.544118 (+ 503us) server_negotiation.cc:407] Connection header received
0808 23:29:04.544279 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:04.544282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:04.544385 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:04.544459 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:04.546441 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.547404 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.549399 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.550263 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.550377 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:04.553862 (+ 3485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:04.553867 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:04.553870 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:04.553898 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:04.557518 (+ 3620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:04.558452 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:04.558458 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:04.558461 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:04.558580 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:04.559049 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:04.559053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:04.559055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:04.559341 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:04.559516 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:04.560065 (+ 549us) server_negotiation.cc:299] Negotiation successful
0808 23:29:04.560247 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 23:29:04.565043 27276 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:29:04.575929 27276 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:29:04.621884 28073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:04.592469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:45213 (local address 127.0.0.1:55354)
0808 23:29:04.605792 (+ 13323us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:04.605831 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:04.606061 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:04.610131 (+ 4070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:04.610140 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:04.610170 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:29:04.611311 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:04.611327 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.613818 (+ 2491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.613824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:04.617824 (+ 4000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:04.617838 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.620515 (+ 2677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.620524 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:04.620792 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:04.620849 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 23:29:04.620981 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 23:29:04.621163 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12992,"mutex_wait_us":3,"thread_start_us":12732,"threads_started":1}
I20260808 23:29:04.622248 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:04.594597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50482 (local address 127.9.44.65:37161)
0808 23:29:04.605798 (+ 11201us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:04.605809 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:04.609707 (+ 3898us) server_negotiation.cc:407] Connection header received
0808 23:29:04.610026 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:04.610033 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:04.610244 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:04.610511 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:29:04.612667 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.614855 (+ 2188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.618486 (+ 3631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.620473 (+ 1987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.620598 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:04.620973 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:04.621416 (+ 443us) server_negotiation.cc:299] Negotiation successful
0808 23:29:04.621587 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10986,"thread_start_us":10700,"threads_started":1}
I20260808 23:29:04.623052 28079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:04.593355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55354 (local address 127.9.44.66:45213)
0808 23:29:04.608762 (+ 15407us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:04.608770 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:04.608804 (+ 34us) server_negotiation.cc:407] Connection header received
0808 23:29:04.608906 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:04.608915 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:04.609220 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:04.609458 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:29:04.611558 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.613662 (+ 2104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.617969 (+ 4307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.619854 (+ 1885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.619984 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:04.620282 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:04.622264 (+ 1982us) server_negotiation.cc:299] Negotiation successful
0808 23:29:04.622496 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15183,"thread_start_us":14919,"threads_started":1}
I20260808 23:29:04.622217 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:04.593551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:37161 (local address 127.0.0.1:50482)
0808 23:29:04.609589 (+ 16038us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:04.609608 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:04.609778 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:04.611030 (+ 1252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:04.611035 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:04.611068 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:29:04.612370 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:04.612384 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.615022 (+ 2638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.615027 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:04.618276 (+ 3249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:04.618293 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.620928 (+ 2635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.620937 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:04.621202 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:04.621238 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 23:29:04.621322 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 23:29:04.621447 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15804,"mutex_wait_us":129,"thread_start_us":12974,"threads_started":1}
I20260808 23:29:04.623971 28080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:04.592526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:42923 (local address 127.0.0.1:35692)
0808 23:29:04.606990 (+ 14464us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:04.607010 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:04.607189 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:04.611588 (+ 4399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:04.611593 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:04.611621 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:29:04.612778 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:04.612794 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.616966 (+ 4172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.616971 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:04.620181 (+ 3210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:04.620195 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.622804 (+ 2609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.622812 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:04.623052 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:04.623077 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 23:29:04.623155 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 23:29:04.623279 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14296,"mutex_wait_us":23,"thread_start_us":16562,"threads_started":1}
I20260808 23:29:04.624205 28077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:04.592669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35692 (local address 127.9.44.67:42923)
0808 23:29:04.611049 (+ 18380us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:04.611055 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:04.611102 (+ 47us) server_negotiation.cc:407] Connection header received
0808 23:29:04.611175 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:04.611180 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:04.611369 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:04.612932 (+ 1563us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:29:04.613610 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.616797 (+ 3187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.620359 (+ 3562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.622531 (+ 2172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.622658 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:04.622981 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:04.623235 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 23:29:04.623447 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18120,"thread_start_us":17819,"threads_started":1}
I20260808 23:29:04.628209 27965 tablet_service.cc:1467] Processing CreateTablet for tablet 7409746322374ef3a5c831a322b31946 (DEFAULT_TABLE table=test-table [id=7cf58c26c1e04c8c94cdd89a09960917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:29:04.628118 27756 tablet_service.cc:1467] Processing CreateTablet for tablet 7409746322374ef3a5c831a322b31946 (DEFAULT_TABLE table=test-table [id=7cf58c26c1e04c8c94cdd89a09960917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:29:04.628098 27539 tablet_service.cc:1467] Processing CreateTablet for tablet 7409746322374ef3a5c831a322b31946 (DEFAULT_TABLE table=test-table [id=7cf58c26c1e04c8c94cdd89a09960917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:29:04.629758 27756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7409746322374ef3a5c831a322b31946. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:29:04.629781 27965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7409746322374ef3a5c831a322b31946. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:29:04.629894 27539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7409746322374ef3a5c831a322b31946. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:29:04.645320 28082 tablet_bootstrap.cc:492] T 7409746322374ef3a5c831a322b31946 P 6fc86ab343dd40fd9ab697e60dab1a08: Bootstrap starting.
I20260808 23:29:04.650888 28082 tablet_bootstrap.cc:654] T 7409746322374ef3a5c831a322b31946 P 6fc86ab343dd40fd9ab697e60dab1a08: Neither blocks nor log segments found. Creating new log.
I20260808 23:29:04.652465 28085 tablet_bootstrap.cc:492] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c: Bootstrap starting.
I20260808 23:29:04.653086 28082 log.cc:824] T 7409746322374ef3a5c831a322b31946 P 6fc86ab343dd40fd9ab697e60dab1a08: Log is configured to *not* fsync() on all Append() calls
I20260808 23:29:04.653170 28084 tablet_bootstrap.cc:492] T 7409746322374ef3a5c831a322b31946 P dc5325b3607345aa9760768125671bc2: Bootstrap starting.
I20260808 23:29:04.657986 28084 tablet_bootstrap.cc:654] T 7409746322374ef3a5c831a322b31946 P dc5325b3607345aa9760768125671bc2: Neither blocks nor log segments found. Creating new log.
I20260808 23:29:04.658097 28085 tablet_bootstrap.cc:654] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c: Neither blocks nor log segments found. Creating new log.
I20260808 23:29:04.659802 28085 log.cc:824] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c: Log is configured to *not* fsync() on all Append() calls
I20260808 23:29:04.660189 28084 log.cc:824] T 7409746322374ef3a5c831a322b31946 P dc5325b3607345aa9760768125671bc2: Log is configured to *not* fsync() on all Append() calls
I20260808 23:29:04.661393 28082 tablet_bootstrap.cc:492] T 7409746322374ef3a5c831a322b31946 P 6fc86ab343dd40fd9ab697e60dab1a08: No bootstrap required, opened a new log
I20260808 23:29:04.661657 28082 ts_tablet_manager.cc:1397] T 7409746322374ef3a5c831a322b31946 P 6fc86ab343dd40fd9ab697e60dab1a08: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.008s
I20260808 23:29:04.664260 28085 tablet_bootstrap.cc:492] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c: No bootstrap required, opened a new log
I20260808 23:29:04.664656 28085 ts_tablet_manager.cc:1397] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.009s
I20260808 23:29:04.665524 28084 tablet_bootstrap.cc:492] T 7409746322374ef3a5c831a322b31946 P dc5325b3607345aa9760768125671bc2: No bootstrap required, opened a new log
I20260808 23:29:04.665808 28084 ts_tablet_manager.cc:1397] T 7409746322374ef3a5c831a322b31946 P dc5325b3607345aa9760768125671bc2: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.008s
I20260808 23:29:04.677616 28084 raft_consensus.cc:348] T 7409746322374ef3a5c831a322b31946 P dc5325b3607345aa9760768125671bc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc86ab343dd40fd9ab697e60dab1a08" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 42923 } } peers { permanent_uuid: "4d015d84d7d347d29b15304f2c44636c" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45213 } } peers { permanent_uuid: "dc5325b3607345aa9760768125671bc2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 37161 } }
I20260808 23:29:04.677502 28082 raft_consensus.cc:348] T 7409746322374ef3a5c831a322b31946 P 6fc86ab343dd40fd9ab697e60dab1a08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc86ab343dd40fd9ab697e60dab1a08" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 42923 } } peers { permanent_uuid: "4d015d84d7d347d29b15304f2c44636c" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45213 } } peers { permanent_uuid: "dc5325b3607345aa9760768125671bc2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 37161 } }
I20260808 23:29:04.678123 28084 raft_consensus.cc:374] T 7409746322374ef3a5c831a322b31946 P dc5325b3607345aa9760768125671bc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:29:04.678179 28082 raft_consensus.cc:374] T 7409746322374ef3a5c831a322b31946 P 6fc86ab343dd40fd9ab697e60dab1a08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:29:04.678344 28084 raft_consensus.cc:729] T 7409746322374ef3a5c831a322b31946 P dc5325b3607345aa9760768125671bc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc5325b3607345aa9760768125671bc2, State: Initialized, Role: FOLLOWER
I20260808 23:29:04.678350 28082 raft_consensus.cc:729] T 7409746322374ef3a5c831a322b31946 P 6fc86ab343dd40fd9ab697e60dab1a08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc86ab343dd40fd9ab697e60dab1a08, State: Initialized, Role: FOLLOWER
I20260808 23:29:04.679024 28084 consensus_queue.cc:260] T 7409746322374ef3a5c831a322b31946 P dc5325b3607345aa9760768125671bc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc86ab343dd40fd9ab697e60dab1a08" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 42923 } } peers { permanent_uuid: "4d015d84d7d347d29b15304f2c44636c" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45213 } } peers { permanent_uuid: "dc5325b3607345aa9760768125671bc2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 37161 } }
I20260808 23:29:04.679163 28082 consensus_queue.cc:260] T 7409746322374ef3a5c831a322b31946 P 6fc86ab343dd40fd9ab697e60dab1a08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc86ab343dd40fd9ab697e60dab1a08" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 42923 } } peers { permanent_uuid: "4d015d84d7d347d29b15304f2c44636c" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45213 } } peers { permanent_uuid: "dc5325b3607345aa9760768125671bc2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 37161 } }
I20260808 23:29:04.681365 28084 ts_tablet_manager.cc:1428] T 7409746322374ef3a5c831a322b31946 P dc5325b3607345aa9760768125671bc2: Time spent starting tablet: real 0.015s user 0.008s sys 0.009s
I20260808 23:29:04.682432 28085 raft_consensus.cc:348] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc86ab343dd40fd9ab697e60dab1a08" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 42923 } } peers { permanent_uuid: "4d015d84d7d347d29b15304f2c44636c" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45213 } } peers { permanent_uuid: "dc5325b3607345aa9760768125671bc2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 37161 } }
I20260808 23:29:04.683142 28085 raft_consensus.cc:374] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:29:04.683465 28058 heartbeater.cc:503] Master 127.9.44.126:33453 was elected leader, sending a full tablet report...
I20260808 23:29:04.683404 28085 raft_consensus.cc:729] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d015d84d7d347d29b15304f2c44636c, State: Initialized, Role: FOLLOWER
I20260808 23:29:04.684047 28082 ts_tablet_manager.cc:1428] T 7409746322374ef3a5c831a322b31946 P 6fc86ab343dd40fd9ab697e60dab1a08: Time spent starting tablet: real 0.022s user 0.022s sys 0.000s
I20260808 23:29:04.684345 28085 consensus_queue.cc:260] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc86ab343dd40fd9ab697e60dab1a08" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 42923 } } peers { permanent_uuid: "4d015d84d7d347d29b15304f2c44636c" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45213 } } peers { permanent_uuid: "dc5325b3607345aa9760768125671bc2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 37161 } }
I20260808 23:29:04.686863 28085 ts_tablet_manager.cc:1428] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c: Time spent starting tablet: real 0.022s user 0.016s sys 0.004s
I20260808 23:29:04.752888 28094 raft_consensus.cc:482] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:29:04.753288 28094 raft_consensus.cc:504] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc86ab343dd40fd9ab697e60dab1a08" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 42923 } } peers { permanent_uuid: "4d015d84d7d347d29b15304f2c44636c" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45213 } } peers { permanent_uuid: "dc5325b3607345aa9760768125671bc2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 37161 } }
W20260808 23:29:04.755339 28059 tablet.cc:2369] T 7409746322374ef3a5c831a322b31946 P 6fc86ab343dd40fd9ab697e60dab1a08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:29:04.756682 28094 leader_election.cc:288] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fc86ab343dd40fd9ab697e60dab1a08 (127.9.44.67:42923), dc5325b3607345aa9760768125671bc2 (127.9.44.65:37161)
W20260808 23:29:04.766234 27865 tablet.cc:2369] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:29:04.767616 28077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:04.756439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:52725 (local address 127.9.44.67:42923)
0808 23:29:04.756647 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:04.756651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:04.758740 (+ 2089us) server_negotiation.cc:407] Connection header received
0808 23:29:04.758976 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:04.758983 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:04.759050 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:04.759114 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:29:04.760533 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.761479 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.764349 (+ 2870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.766112 (+ 1763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.766213 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:04.766331 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:04.767212 (+ 881us) server_negotiation.cc:299] Negotiation successful
0808 23:29:04.767346 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 23:29:04.768052 28096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:04.756557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:42923 (local address 127.9.44.66:52725)
0808 23:29:04.758624 (+ 2067us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:04.758642 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:04.758794 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:04.759154 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:04.759159 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:04.759182 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:29:04.760387 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:04.760400 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.761622 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.761628 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:04.764139 (+ 2511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:04.764157 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.766768 (+ 2611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.766777 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:04.767040 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:04.767065 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 23:29:04.767145 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 23:29:04.767233 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1899,"mutex_wait_us":20,"thread_start_us":1296,"threads_started":1}
I20260808 23:29:04.768085 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:04.756096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:40207 (local address 127.9.44.65:37161)
0808 23:29:04.756306 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:04.756312 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:04.758829 (+ 2517us) server_negotiation.cc:407] Connection header received
0808 23:29:04.759015 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:04.759019 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:04.759077 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:04.759162 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:29:04.760886 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.761975 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.764919 (+ 2944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.766368 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.766494 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:04.766633 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:04.767577 (+ 944us) server_negotiation.cc:299] Negotiation successful
0808 23:29:04.767748 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 23:29:04.768594 28097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:04.756750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:37161 (local address 127.9.44.66:40207)
0808 23:29:04.758676 (+ 1926us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:04.758722 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:04.758879 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:04.759189 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:04.759194 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:04.759227 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:29:04.760612 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:04.760627 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.762134 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.762138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:04.764762 (+ 2624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:04.764775 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.767160 (+ 2385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.767171 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:04.767390 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:04.767430 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 23:29:04.767522 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 23:29:04.767639 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1783,"mutex_wait_us":4,"thread_start_us":1358,"threads_started":1}
I20260808 23:29:04.769547 27995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7409746322374ef3a5c831a322b31946" candidate_uuid: "4d015d84d7d347d29b15304f2c44636c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc86ab343dd40fd9ab697e60dab1a08" is_pre_election: true
I20260808 23:29:04.769964 27567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7409746322374ef3a5c831a322b31946" candidate_uuid: "4d015d84d7d347d29b15304f2c44636c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc5325b3607345aa9760768125671bc2" is_pre_election: true
I20260808 23:29:04.770282 27995 raft_consensus.cc:2454] T 7409746322374ef3a5c831a322b31946 P 6fc86ab343dd40fd9ab697e60dab1a08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d015d84d7d347d29b15304f2c44636c in term 0.
I20260808 23:29:04.770859 27567 raft_consensus.cc:2454] T 7409746322374ef3a5c831a322b31946 P dc5325b3607345aa9760768125671bc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d015d84d7d347d29b15304f2c44636c in term 0.
I20260808 23:29:04.771229 27697 leader_election.cc:302] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d015d84d7d347d29b15304f2c44636c, 6fc86ab343dd40fd9ab697e60dab1a08; no voters:
I20260808 23:29:04.771802 28094 raft_consensus.cc:2781] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:29:04.772048 28094 raft_consensus.cc:482] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:29:04.772238 28094 raft_consensus.cc:3037] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:29:04.775584 28094 raft_consensus.cc:504] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc86ab343dd40fd9ab697e60dab1a08" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 42923 } } peers { permanent_uuid: "4d015d84d7d347d29b15304f2c44636c" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45213 } } peers { permanent_uuid: "dc5325b3607345aa9760768125671bc2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 37161 } }
I20260808 23:29:04.776501 28094 leader_election.cc:288] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c [CANDIDATE]: Term 1 election: Requested vote from peers 6fc86ab343dd40fd9ab697e60dab1a08 (127.9.44.67:42923), dc5325b3607345aa9760768125671bc2 (127.9.44.65:37161)
I20260808 23:29:04.776968 27995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7409746322374ef3a5c831a322b31946" candidate_uuid: "4d015d84d7d347d29b15304f2c44636c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc86ab343dd40fd9ab697e60dab1a08"
I20260808 23:29:04.777079 27567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7409746322374ef3a5c831a322b31946" candidate_uuid: "4d015d84d7d347d29b15304f2c44636c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc5325b3607345aa9760768125671bc2"
I20260808 23:29:04.777261 27995 raft_consensus.cc:3037] T 7409746322374ef3a5c831a322b31946 P 6fc86ab343dd40fd9ab697e60dab1a08 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:29:04.777333 27567 raft_consensus.cc:3037] T 7409746322374ef3a5c831a322b31946 P dc5325b3607345aa9760768125671bc2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:29:04.782245 27995 raft_consensus.cc:2454] T 7409746322374ef3a5c831a322b31946 P 6fc86ab343dd40fd9ab697e60dab1a08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d015d84d7d347d29b15304f2c44636c in term 1.
I20260808 23:29:04.782222 27567 raft_consensus.cc:2454] T 7409746322374ef3a5c831a322b31946 P dc5325b3607345aa9760768125671bc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d015d84d7d347d29b15304f2c44636c in term 1.
I20260808 23:29:04.782775 27696 leader_election.cc:302] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d015d84d7d347d29b15304f2c44636c, dc5325b3607345aa9760768125671bc2; no voters:
I20260808 23:29:04.783238 28094 raft_consensus.cc:2781] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:29:04.784169 28094 raft_consensus.cc:686] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c [term 1 LEADER]: Becoming Leader. State: Replica: 4d015d84d7d347d29b15304f2c44636c, State: Running, Role: LEADER
I20260808 23:29:04.784776 28094 consensus_queue.cc:237] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc86ab343dd40fd9ab697e60dab1a08" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 42923 } } peers { permanent_uuid: "4d015d84d7d347d29b15304f2c44636c" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45213 } } peers { permanent_uuid: "dc5325b3607345aa9760768125671bc2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 37161 } }
I20260808 23:29:04.790431 27274 catalog_manager.cc:5495] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d015d84d7d347d29b15304f2c44636c (127.9.44.66). New cstate: current_term: 1 leader_uuid: "4d015d84d7d347d29b15304f2c44636c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fc86ab343dd40fd9ab697e60dab1a08" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 42923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d015d84d7d347d29b15304f2c44636c" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 45213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dc5325b3607345aa9760768125671bc2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 37161 } health_report { overall_health: UNKNOWN } } }
I20260808 23:29:04.811933 28079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:04.805763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55360 (local address 127.9.44.66:45213)
0808 23:29:04.805982 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:04.805986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:04.806002 (+ 16us) server_negotiation.cc:407] Connection header received
0808 23:29:04.806069 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:04.806072 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:04.806138 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:04.806253 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:29:04.806756 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.807882 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.809889 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.810426 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.810518 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:04.810977 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:29:04.811451 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:29:04.811552 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:04.811624 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 23:29:04.811714 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 23:29:04.822872 27567 raft_consensus.cc:1261] T 7409746322374ef3a5c831a322b31946 P dc5325b3607345aa9760768125671bc2 [term 1 FOLLOWER]: Refusing update from remote peer 4d015d84d7d347d29b15304f2c44636c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:29:04.822871 27995 raft_consensus.cc:1261] T 7409746322374ef3a5c831a322b31946 P 6fc86ab343dd40fd9ab697e60dab1a08 [term 1 FOLLOWER]: Refusing update from remote peer 4d015d84d7d347d29b15304f2c44636c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:29:04.823939 28098 consensus_queue.cc:1035] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fc86ab343dd40fd9ab697e60dab1a08" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 42923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:29:04.824519 28094 consensus_queue.cc:1035] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc5325b3607345aa9760768125671bc2" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 37161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:29:04.839731 28101 mvcc.cc:205] Tried to move back new op lower bound from 7316405226783039488 to 7316405226643308544. Current Snapshot: MvccSnapshot[applied={T|T < 7316405226783039488}]
I20260808 23:29:04.859200 27274 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:29:04.859558 27274 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:29:04.863983 27274 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d5e04849307d482bba87bee9ca4cb68e: Sending DeleteTablet for 3 replicas of tablet 7409746322374ef3a5c831a322b31946
I20260808 23:29:04.865408 27965 tablet_service.cc:1514] Processing DeleteTablet for tablet 7409746322374ef3a5c831a322b31946 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:29:04 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:35692
I20260808 23:29:04.865497 27756 tablet_service.cc:1514] Processing DeleteTablet for tablet 7409746322374ef3a5c831a322b31946 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:29:04 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:55354
I20260808 23:29:04.865694 27539 tablet_service.cc:1514] Processing DeleteTablet for tablet 7409746322374ef3a5c831a322b31946 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:29:04 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:50482
I20260808 23:29:04.868314 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 27412
I20260808 23:29:04.877413 28120 tablet_replica.cc:330] T 7409746322374ef3a5c831a322b31946 P 6fc86ab343dd40fd9ab697e60dab1a08: stopping tablet replica
I20260808 23:29:04.878041 28120 raft_consensus.cc:2229] T 7409746322374ef3a5c831a322b31946 P 6fc86ab343dd40fd9ab697e60dab1a08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:29:04.879055 28120 raft_consensus.cc:2258] T 7409746322374ef3a5c831a322b31946 P 6fc86ab343dd40fd9ab697e60dab1a08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:29:04.882673 28121 tablet_replica.cc:330] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c: stopping tablet replica
I20260808 23:29:04.883404 28121 raft_consensus.cc:2229] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:29:04.884577 28121 raft_consensus.cc:2258] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:29:04.887161 28120 ts_tablet_manager.cc:1905] T 7409746322374ef3a5c831a322b31946 P 6fc86ab343dd40fd9ab697e60dab1a08: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 23:29:04 dist-test-slave-6bbx krb5kdc[26523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.65@KRBTEST.COM
I20260808 23:29:04.894629 28121 ts_tablet_manager.cc:1905] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:29:04.898000 28122 tablet_replica.cc:330] T 7409746322374ef3a5c831a322b31946 P dc5325b3607345aa9760768125671bc2: stopping tablet replica
I20260808 23:29:04.898485 28120 ts_tablet_manager.cc:1918] T 7409746322374ef3a5c831a322b31946 P 6fc86ab343dd40fd9ab697e60dab1a08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:29:04.898597 28122 raft_consensus.cc:2229] T 7409746322374ef3a5c831a322b31946 P dc5325b3607345aa9760768125671bc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:29:04.898733 28120 log.cc:1196] T 7409746322374ef3a5c831a322b31946 P 6fc86ab343dd40fd9ab697e60dab1a08: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/7409746322374ef3a5c831a322b31946
I20260808 23:29:04.899461 28120 ts_tablet_manager.cc:1939] T 7409746322374ef3a5c831a322b31946 P 6fc86ab343dd40fd9ab697e60dab1a08: Deleting consensus metadata
I20260808 23:29:04.901713 27255 catalog_manager.cc:4841] TS 6fc86ab343dd40fd9ab697e60dab1a08 (127.9.44.67:42923): tablet 7409746322374ef3a5c831a322b31946 (table test-table [id=7cf58c26c1e04c8c94cdd89a09960917]) successfully deleted
I20260808 23:29:04.903905 28122 raft_consensus.cc:2258] T 7409746322374ef3a5c831a322b31946 P dc5325b3607345aa9760768125671bc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:29:04.906215 28122 ts_tablet_manager.cc:1905] T 7409746322374ef3a5c831a322b31946 P dc5325b3607345aa9760768125671bc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:29:04.906919 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:04.869735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50484 (local address 127.9.44.65:37161)
0808 23:29:04.869909 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:04.869915 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:04.883345 (+ 13430us) server_negotiation.cc:407] Connection header received
0808 23:29:04.883917 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:04.883923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:04.884035 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:04.884114 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:04.886169 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.887452 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.889465 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:04.890180 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:04.890279 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:04.896063 (+ 5784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:04.896080 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:04.896098 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:04.896152 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:04.903140 (+ 6988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:04.904112 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:04.904133 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:04.904136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:04.904302 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:04.904787 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:04.904792 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:04.904795 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:04.905307 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:04.905612 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:04.906261 (+ 649us) server_negotiation.cc:299] Negotiation successful
0808 23:29:04.906556 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 23:29:04.907435 28121 ts_tablet_manager.cc:1918] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:29:04.907701 28121 log.cc:1196] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/7409746322374ef3a5c831a322b31946
I20260808 23:29:04.908555 28121 ts_tablet_manager.cc:1939] T 7409746322374ef3a5c831a322b31946 P 4d015d84d7d347d29b15304f2c44636c: Deleting consensus metadata
I20260808 23:29:04.908876 27636 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:29:04.910909 27253 catalog_manager.cc:4841] TS 4d015d84d7d347d29b15304f2c44636c (127.9.44.66:45213): tablet 7409746322374ef3a5c831a322b31946 (table test-table [id=7cf58c26c1e04c8c94cdd89a09960917]) successfully deleted
I20260808 23:29:05.000331 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 27412
I20260808 23:29:05.021644 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 27649
W20260808 23:29:05.021550 27254 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.44.65:37161 (error 108)
I20260808 23:29:05.023993 28080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:05.023563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:37161 (local address 127.0.0.1:50488)
0808 23:29:05.023751 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:05.023852 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:37161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 23:29:05.024282 27254 catalog_manager.cc:4584] TS dc5325b3607345aa9760768125671bc2 (127.9.44.65:37161): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7409746322374ef3a5c831a322b31946: Network error: Client connection negotiation failed: client connection to 127.9.44.65:37161: connect: Connection refused (error 111)
Aug 08 23:29:05 dist-test-slave-6bbx krb5kdc[26523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.66@KRBTEST.COM
I20260808 23:29:05.044859 28079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:05.022355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55372 (local address 127.9.44.66:45213)
0808 23:29:05.022541 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:05.022547 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:05.022898 (+ 351us) server_negotiation.cc:407] Connection header received
0808 23:29:05.022982 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:05.022988 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:05.023086 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:05.023212 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:05.025622 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:05.026425 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:05.028998 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:05.029437 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:05.029538 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:05.038642 (+ 9104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:05.038658 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:05.038675 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:05.038723 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:05.042061 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:05.042714 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:05.042722 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:05.042724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:05.042931 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:05.043309 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:05.043312 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:05.043314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:05.043696 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:05.043986 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:05.044308 (+ 322us) server_negotiation.cc:299] Negotiation successful
0808 23:29:05.044539 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 23:29:05.045526 27853 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:29:05.078099 28080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:05.077495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:37161 (local address 127.0.0.1:50500)
0808 23:29:05.077779 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:05.077896 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:37161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260808 23:29:05.079420 28080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:05.078914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:37161 (local address 127.0.0.1:50512)
0808 23:29:05.079156 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:05.079250 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:37161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 23:29:05.140102 28080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:05.139614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:37161 (local address 127.0.0.1:50520)
0808 23:29:05.139848 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:05.139942 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:37161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260808 23:29:05.141090 28080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:05.140768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:37161 (local address 127.0.0.1:50534)
0808 23:29:05.140900 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:05.140972 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:37161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 23:29:05.157626 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 27649
I20260808 23:29:05.186205 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 27872
Aug 08 23:29:05 dist-test-slave-6bbx krb5kdc[26523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.67@KRBTEST.COM
I20260808 23:29:05.200194 28077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:05.186963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35694 (local address 127.9.44.67:42923)
0808 23:29:05.187140 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:05.187145 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:05.187168 (+ 23us) server_negotiation.cc:407] Connection header received
0808 23:29:05.187329 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:05.187333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:05.187409 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:05.187490 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:05.188777 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:05.189509 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:05.190713 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:05.191078 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:05.191164 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:05.195130 (+ 3966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:05.195144 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:05.195159 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:05.195191 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:05.197837 (+ 2646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:05.198496 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:05.198502 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:05.198504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:05.198638 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:05.198952 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:05.198955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:05.198957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:05.199304 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:05.199530 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:05.199779 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 23:29:05.199942 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 23:29:05.200789 28051 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:29:05.221524 28080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:05.221096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:37161 (local address 127.0.0.1:50538)
0808 23:29:05.221284 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:05.221367 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:37161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 23:29:05.222550 28080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:05.222177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:37161 (local address 127.0.0.1:50548)
0808 23:29:05.222371 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:05.222441 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:37161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260808 23:29:05.288599 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 27872
I20260808 23:29:05.311084 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 27233
I20260808 23:29:05.312258 27333 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:29:05.425369 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 27233
I20260808 23:29:05.425395 28080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:05.424911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:37161 (local address 127.0.0.1:50556)
0808 23:29:05.425071 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:05.425178 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:37161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 23:29:05.455154 9393 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 23:29:05.788704 9393 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 23:29:05.788949 9393 mini_ranger.cc:66] Stopping Ranger...
I20260808 23:29:06.132719 9393 mini_ranger.cc:68] Stopped Ranger
2026-08-08 23:29:06.132 UTC [26563] LOG: received smart shutdown request
2026-08-08 23:29:06.136 UTC [26563] LOG: background worker "logical replication launcher" (PID 26579) exited with exit code 1
2026-08-08 23:29:06.136 UTC [26569] LOG: shutting down
2026-08-08 23:29:06.253 UTC [26563] LOG: database system is shut down
2026-08-08T23:29:06Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48844 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:29:06 dist-test-slave-6bbx krb5kdc[28140](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:29:06 dist-test-slave-6bbx krb5kdc[28140](info): set up 2 sockets
Aug 08 23:29:06 dist-test-slave-6bbx krb5kdc[28140](info): commencing operation
krb5kdc: starting...
W20260808 23:29:07.561251 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:29:07 dist-test-slave-6bbx krb5kdc[28140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:29:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:29:07Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.44.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.44.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.44.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/rangeradmin_127.9.44.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.44.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/rangeradmin_127.9.44.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.44.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.44.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.44.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/rangerlookup_127.9.44.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.44.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/rangerlookup_127.9.44.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.44.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.44.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.44.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/HTTP_127.9.44.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.44.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/HTTP_127.9.44.84@KRBTEST.COM.keytab.
I20260808 23:29:07.858891 9393 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskzx8VIc/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/postgres -l logfile start
2026-08-08 23:29:09.957 UTC [28188] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 23:29:09.958 UTC [28188] LOG: listening on IPv4 address "127.9.44.84", port 59997
2026-08-08 23:29:09.960 UTC [28188] LOG: listening on Unix socket "/tmp/.s.PGSQL.59997"
2026-08-08 23:29:09.981 UTC [28193] LOG: database system was shut down at 2026-08-08 23:29:08 UTC
2026-08-08 23:29:09.984 UTC [28188] LOG: database system is ready to accept connections
I20260808 23:29:11.047982 9393 mini_postgres.cc:96] Postgres bound to 59997
2026-08-08 23:29:11.069 UTC [28207] FATAL: database "slave" does not exist
127.9.44.84:59997 - accepting connections
I20260808 23:29:11.070140 9393 mini_ranger.cc:161] Starting Ranger...
I20260808 23:29:11.097672 9393 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 23:29:11.462469 9393 mini_ranger.cc:87] Created ranger Postgres database
I20260808 23:29:11.462586 9393 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ranger-admin
2026-08-08 23:29:11,739 [I] DB FLAVOR :POSTGRES
2026-08-08 23:29:11,739 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 23:29:11,740 [I] Checking connection..
2026-08-08 23:29:11,740 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:59997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 23:29:11,981 [I] Checking connection passed.
2026-08-08 23:29:11,981 [I] --------- Verifying version history table ---------
2026-08-08 23:29:11,981 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:59997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 23:29:12,233 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 23:29:12,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:59997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 23:29:12,486 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 23:29:12,486 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 23:29:12,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:59997/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 23:29:12.733 UTC [28307] WARNING: there is no transaction in progress
2026-08-08 23:29:12,741 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 23:29:12,741 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:59997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 23:29:12,978 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 23:29:12,978 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 23:29:12,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:59997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 23:29:13,230 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:59997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 23:29:13,474 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:59997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6bbx.c.gcp-upstream.internal','N') ;"
2026-08-08 23:29:13,720 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:59997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 23:29:13,977 [I] Table x_portal_user does not exist in database ranger
2026-08-08 23:29:13,977 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:59997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 23:29:14,217 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 23:29:14,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:59997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 23:29:14,461 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:59997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 23:29:14,698 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 23:29:14,699 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:59997/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 23:29:15.550 UTC [28469] WARNING: there is no transaction in progress
2026-08-08 23:29:15.564 UTC [28469] WARNING: there is no transaction in progress
2026-08-08 23:29:15.577 UTC [28469] WARNING: there is no transaction in progress
2026-08-08 23:29:15.590 UTC [28469] WARNING: there is no transaction in progress
2026-08-08 23:29:15.606 UTC [28469] WARNING: there is no transaction in progress
2026-08-08 23:29:15.622 UTC [28469] WARNING: there is no transaction in progress
2026-08-08 23:29:15.670 UTC [28469] WARNING: there is no transaction in progress
2026-08-08 23:29:15.675 UTC [28469] WARNING: there is no transaction in progress
2026-08-08 23:29:15.687 UTC [28469] WARNING: there is no transaction in progress
2026-08-08 23:29:15.695 UTC [28469] WARNING: there is no transaction in progress
2026-08-08 23:29:15.704 UTC [28469] WARNING: there is no transaction in progress
2026-08-08 23:29:15.716 UTC [28469] WARNING: there is no transaction in progress
2026-08-08 23:29:15.724 UTC [28469] WARNING: there is no transaction in progress
2026-08-08 23:29:15.729 UTC [28469] WARNING: there is no transaction in progress
2026-08-08 23:29:15.734 UTC [28469] WARNING: there is no transaction in progress
2026-08-08 23:29:15.740 UTC [28469] WARNING: there is no transaction in progress
2026-08-08 23:29:16,250 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 23:29:16,250 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:59997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 23:29:16,493 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 23:29:16,493 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:59997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 23:29:16,746 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 23:29:16,747 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:59997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 23:29:17,002 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 23:29:17,003 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:59997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 23:29:17,245 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:59997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 23:29:17,495 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:59997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6bbx.c.gcp-upstream.internal';"
2026-08-08 23:29:17,740 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 23:29:17,741 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:59997/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 23:29:17,963 [I] DB_PATCHES have already been applied
I20260808 23:29:17.969647 9393 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ranger-admin:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskzx8VIc/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 23:29:17.969734 9393 mini_ranger.cc:193] Using host: 127.9.44.84
I20260808 23:29:17.988377 9393 mini_ranger.cc:227] Ranger admin URL: http://127.9.44.84:49943
Aug 08, 2026 11:29:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 11:29:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 11:29:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 11:29:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 11:29:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 11:29:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 11:29:18 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 23:29:19.206386 9393 mini_ranger.cc:160] Time spent starting Ranger: real 8.136s user 0.001s sys 0.050s
Aug 08, 2026 11:29:37 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 11:29:37 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 11:29:37 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 11:29:37 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 11:29:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 11:29:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 11:29:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 11:29:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 11:29:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 11:29:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 11:29:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 11:29:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 11:29:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 11:29:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 11:29:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 11:29:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 11:29:37 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 11:29:38 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 23:29:38.897075 9393 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.44.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.44.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.44.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/rangerkms_127.9.44.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.44.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/rangerkms_127.9.44.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.44.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.44.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.44.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/HTTP_127.9.44.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.44.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/HTTP_127.9.44.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 23:29:39 dist-test-slave-6bbx krb5kdc[28140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231779, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 23:29:39.054553 9393 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 23:29:39.074167 9393 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 23:29:39.691615 9393 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 23:29:39.691764 9393 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ranger-kms
I20260808 23:29:39.691819 9393 mini_ranger_kms.cc:227] Using postgres at 127.9.44.84:59997
2026-08-08 23:29:40,037 [I] DB FLAVOR :POSTGRES
2026-08-08 23:29:40,037 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 23:29:40,037 [I] Checking connection
2026-08-08 23:29:40,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:59997/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 23:29:40,300 [I] connection success
2026-08-08 23:29:40,300 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 23:29:40,301 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 23:29:40,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:59997/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 23:29:40,542 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 23:29:40,542 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 23:29:40,542 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 23:29:40,542 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.44.84:59997/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 23:29:40,824 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 23:29:41.249578 9393 mini_ranger_kms.cc:310] Created kms service
I20260808 23:29:41.440934 9393 mini_ranger_kms.cc:326] Created kudu user
I20260808 23:29:41.480504 9393 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 23:29:41 dist-test-slave-6bbx krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231779, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.44.84@KRBTEST.COM
I20260808 23:29:41.848696 9393 mini_ranger_kms.cc:384] Added ranger policy
I20260808 23:29:41.849030 9393 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ranger-kms:/tmp/dist-test-taskzx8VIc/build/asan/bin/postgresql.jar:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskzx8VIc/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 23:29:41.849104 9393 mini_ranger_kms.cc:240] Using host: 127.9.44.84
I20260808 23:29:41.875633 9393 mini_ranger_kms.cc:276] Ranger KMS PID: 28751
I20260808 23:29:41.875757 9393 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.44.84:34085
Aug 08, 2026 11:29:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 11:29:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 11:29:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 11:29:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 11:29:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 11:29:43 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 23:29:44 dist-test-slave-6bbx krb5kdc[28140](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786231784, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.44.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:29:44.349736 9393 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.295s user 0.045s sys 0.329s
I20260808 23:29:44.349989 9393 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 23:29:44.350155 9393 mini_ranger_kms.cc:401] 127.9.44.84:34085/kms/v1/keys
Aug 08 23:29:44 dist-test-slave-6bbx krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231784, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.44.84@KRBTEST.COM for HTTP/127.9.44.84@KRBTEST.COM
Aug 08 23:29:44 dist-test-slave-6bbx krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231784, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.44.84@KRBTEST.COM for HTTP/127.9.44.84@KRBTEST.COM
Aug 08 23:29:47 dist-test-slave-6bbx krb5kdc[28140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:29:47.308760 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:39711
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:36417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:39711
--ranger_config_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.44.84:34085/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:29:47.626513 28819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:29:47.626788 28819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:29:47.626914 28819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:29:47.633700 28819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:29:47.633827 28819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:29:47.633903 28819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:29:47.633973 28819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:29:47.634035 28819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:29:47.643392 28819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:36417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.44.84:34085/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:39711
--ranger_config_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:39711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:29:47.644472 28819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:29:47.645663 28819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:29:47.656633 28831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:29:47.656848 28828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:29:47.656652 28829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:29:47.657413 28819 server_base.cc:1034] running on GCE node
I20260808 23:29:47.658079 28819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:29:47.659317 28819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:29:47.660509 28819 hybrid_clock.cc:648] HybridClock initialized: now 1786231787660466 us; error 29 us; skew 500 ppm
I20260808 23:29:47.660820 28819 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:29:47 dist-test-slave-6bbx krb5kdc[28140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231787, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:29:47.666716 28819 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:29:47.668057 28819 webserver.cc:460] Webserver started at http://127.9.44.126:45243/ using document root <none> and password file <none>
I20260808 23:29:47.668632 28819 fs_manager.cc:359] Metadata directory not provided
I20260808 23:29:47.668741 28819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:29:47.669018 28819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 23:29:47 dist-test-slave-6bbx krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231787, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for HTTP/127.9.44.84@KRBTEST.COM
I20260808 23:29:47.717403 28819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "bdc6add1d3f14011aea938feb0d08f39"
format_stamp: "Formatted at 2026-08-08 23:29:47 on dist-test-slave-6bbx"
server_key: "16d1e8e58baa7b267f719cb2bba061f8"
server_key_iv: "bb12b3743a4b8a72426a9c59982484e0"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:47.718482 28819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "bdc6add1d3f14011aea938feb0d08f39"
format_stamp: "Formatted at 2026-08-08 23:29:47 on dist-test-slave-6bbx"
server_key: "16d1e8e58baa7b267f719cb2bba061f8"
server_key_iv: "bb12b3743a4b8a72426a9c59982484e0"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:47.725428 28819 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260808 23:29:47.742858 28840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:47.744261 28819 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:29:47.744438 28819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "bdc6add1d3f14011aea938feb0d08f39"
format_stamp: "Formatted at 2026-08-08 23:29:47 on dist-test-slave-6bbx"
server_key: "16d1e8e58baa7b267f719cb2bba061f8"
server_key_iv: "bb12b3743a4b8a72426a9c59982484e0"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:47.744635 28819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:29:47.780418 28819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:29:47.784149 28819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:29:47.784518 28819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:29:47.801856 28819 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:39711
I20260808 23:29:47.801873 28918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:39711 every 8 connection(s)
I20260808 23:29:47.803102 28819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:29:47.803434 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 28819
I20260808 23:29:47.803624 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:29:47.804452 9393 external_mini_cluster.cc:1428] Setting key 3cfbc2cfa180510c555bb698918a4bd2
I20260808 23:29:47.807258 28919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 23:29:47 dist-test-slave-6bbx krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:29:47.817494 28919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39: Bootstrap starting.
I20260808 23:29:47.820632 28919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39: Neither blocks nor log segments found. Creating new log.
I20260808 23:29:47.821648 28919 log.cc:824] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39: Log is configured to *not* fsync() on all Append() calls
I20260808 23:29:47.824774 28919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39: No bootstrap required, opened a new log
I20260808 23:29:47.825210 28925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:47.806310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48422 (local address 127.9.44.126:39711)
0808 23:29:47.807055 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:47.807066 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:47.807107 (+ 41us) server_negotiation.cc:407] Connection header received
0808 23:29:47.808727 (+ 1620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:47.808750 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:47.809093 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:47.809444 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:47.810752 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:47.812133 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:47.813428 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:47.814053 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:47.814224 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:47.818151 (+ 3927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:47.818160 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:47.818174 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:47.818207 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:47.821082 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:47.821718 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:47.821724 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:47.821730 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:47.821907 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:47.822369 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:47.822373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:47.822375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:47.822931 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:47.823204 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:47.823571 (+ 367us) server_negotiation.cc:299] Negotiation successful
0808 23:29:47.823980 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":359,"threads_started":1}
I20260808 23:29:47.831925 28919 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc6add1d3f14011aea938feb0d08f39" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39711 } }
I20260808 23:29:47.832237 28919 raft_consensus.cc:374] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:29:47.832312 28919 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdc6add1d3f14011aea938feb0d08f39, State: Initialized, Role: FOLLOWER
I20260808 23:29:47.832746 28919 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc6add1d3f14011aea938feb0d08f39" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39711 } }
I20260808 23:29:47.832933 28919 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:29:47.833026 28919 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:29:47.833148 28919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:29:47.835037 28919 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc6add1d3f14011aea938feb0d08f39" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39711 } }
I20260808 23:29:47.835520 28919 leader_election.cc:302] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdc6add1d3f14011aea938feb0d08f39; no voters:
I20260808 23:29:47.836076 28919 leader_election.cc:288] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:29:47.836180 28928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:29:47.836815 28928 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [term 1 LEADER]: Becoming Leader. State: Replica: bdc6add1d3f14011aea938feb0d08f39, State: Running, Role: LEADER
I20260808 23:29:47.837347 28928 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc6add1d3f14011aea938feb0d08f39" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39711 } }
I20260808 23:29:47.837895 28919 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:29:47.840265 28929 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdc6add1d3f14011aea938feb0d08f39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc6add1d3f14011aea938feb0d08f39" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39711 } } }
I20260808 23:29:47.840238 28931 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdc6add1d3f14011aea938feb0d08f39. Latest consensus state: current_term: 1 leader_uuid: "bdc6add1d3f14011aea938feb0d08f39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc6add1d3f14011aea938feb0d08f39" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39711 } } }
I20260808 23:29:47.840767 28929 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [sys.catalog]: This master's current role is: LEADER
I20260808 23:29:47.840775 28931 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [sys.catalog]: This master's current role is: LEADER
I20260808 23:29:47.841729 28939 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:29:47.847375 28939 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:29:47.848096 28919 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 23:29:47.865181 28939 catalog_manager.cc:1347] Generated new cluster ID: 62eb6a53ff8d4ecda5c2e1400b43aa41
I20260808 23:29:47.865315 28939 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:29:47.875147 28939 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:29:47.876417 28939 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:29:47.882583 28939 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39: Generated new TSK 0
I20260808 23:29:47.883258 28939 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.44.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
I20260808 23:29:48.646145 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.44.84:34085/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:0
--local_ip_for_outbound_sockets=127.9.44.65
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:39711
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:36417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:29:48 dist-test-slave-6bbx krb5kdc[28140](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786231788, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 23:29:49.075383 28989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:29:49.075831 28989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:29:49.076011 28989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:29:49.085527 28989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:29:49.085640 28989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:29:49.085819 28989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:29:49.095690 28989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:36417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.44.84:34085/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:39711
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:29:49.096922 28989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:29:49.098273 28989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:29:49.113900 29003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:29:49.114239 29001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:29:49.119801 28989 server_base.cc:1034] running on GCE node
W20260808 23:29:49.120162 29006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:29:49.120759 28989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:29:49.121708 28989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:29:49.122960 28989 hybrid_clock.cc:648] HybridClock initialized: now 1786231789122893 us; error 55 us; skew 500 ppm
I20260808 23:29:49.123430 28989 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:29:49 dist-test-slave-6bbx krb5kdc[28140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231789, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:29:49.127524 28989 init.cc:377] Logged in from keytab as kudu/127.9.44.65@KRBTEST.COM (short username kudu)
I20260808 23:29:49.140939 28989 webserver.cc:460] Webserver started at http://127.9.44.65:35931/ using document root <none> and password file <none>
I20260808 23:29:49.141671 28989 fs_manager.cc:359] Metadata directory not provided
I20260808 23:29:49.141789 28989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:29:49.142083 28989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 23:29:49 dist-test-slave-6bbx krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231789, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for HTTP/127.9.44.84@KRBTEST.COM
I20260808 23:29:49.164669 28989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/data/instance:
uuid: "b665a9d8e0534eb2a5a48ba61d21b5d8"
format_stamp: "Formatted at 2026-08-08 23:29:49 on dist-test-slave-6bbx"
server_key: "aa51e40c9edbf678468623e4ee3ca2f9"
server_key_iv: "4ab87462fe57892c02d5d01c8840ebcd"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:49.165544 28989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance:
uuid: "b665a9d8e0534eb2a5a48ba61d21b5d8"
format_stamp: "Formatted at 2026-08-08 23:29:49 on dist-test-slave-6bbx"
server_key: "aa51e40c9edbf678468623e4ee3ca2f9"
server_key_iv: "4ab87462fe57892c02d5d01c8840ebcd"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:49.171468 28989 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:29:49.188984 29016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:49.190865 28989 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260808 23:29:49.191048 28989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "b665a9d8e0534eb2a5a48ba61d21b5d8"
format_stamp: "Formatted at 2026-08-08 23:29:49 on dist-test-slave-6bbx"
server_key: "aa51e40c9edbf678468623e4ee3ca2f9"
server_key_iv: "4ab87462fe57892c02d5d01c8840ebcd"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:49.191288 28989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:29:49.242364 28989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:29:49.246644 28989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:29:49.246937 28989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:29:49.248055 28989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:29:49.249753 28989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:29:49.249838 28989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:49.249992 28989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:29:49.250119 28989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:49.325098 28989 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:42727
I20260808 23:29:49.325160 29173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:42727 every 8 connection(s)
I20260808 23:29:49.326777 28989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:29:49.333427 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 28989
I20260808 23:29:49.333639 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance
I20260808 23:29:49.334520 9393 external_mini_cluster.cc:1428] Setting key 807bce26b4f1dc526cac09cec41688d3
Aug 08 23:29:49 dist-test-slave-6bbx krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231789, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:29:49.365978 29178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:49.330235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:39711 (local address 127.9.44.65:48533)
0808 23:29:49.342813 (+ 12578us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:49.342864 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:49.345398 (+ 2534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:49.346412 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:49.346425 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:49.347075 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:29:49.347976 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:49.348005 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:49.349778 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:49.349784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:49.351825 (+ 2041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:49.351842 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:49.353058 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:49.353067 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:49.353345 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:49.354104 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:29:49.354137 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:29:49.357592 (+ 3455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:29:49.361465 (+ 3873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:49.361474 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:49.361489 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:49.362014 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:49.362529 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:49.362534 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:49.362537 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:49.362766 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:49.363403 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:29:49.363410 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:29:49.363868 (+ 458us) client_negotiation.cc:769] Sending connection context
0808 23:29:49.364201 (+ 333us) client_negotiation.cc:240] Negotiation successful
0808 23:29:49.364704 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11339,"thread_start_us":11430,"threads_started":1}
I20260808 23:29:49.367062 29177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:49.329601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:48533 (local address 127.9.44.126:39711)
0808 23:29:49.344951 (+ 15350us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:49.344958 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:49.344983 (+ 25us) server_negotiation.cc:407] Connection header received
0808 23:29:49.345900 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:49.345907 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:49.345988 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:49.348217 (+ 2229us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:49.348390 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:49.349572 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:49.352007 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:49.352642 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:49.352765 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:49.357885 (+ 5120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:49.357894 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:49.357899 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:49.357938 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:49.361308 (+ 3370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:49.362202 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:49.362207 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:49.362210 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:49.362388 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:49.362889 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:49.362893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:49.362896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:49.363208 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:49.363336 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:49.366492 (+ 3156us) server_negotiation.cc:299] Negotiation successful
0808 23:29:49.366722 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15213,"thread_start_us":15375,"threads_started":1}
I20260808 23:29:49.369274 29174 heartbeater.cc:347] Connected to a master server at 127.9.44.126:39711
I20260808 23:29:49.369820 29174 heartbeater.cc:464] Registering TS with master...
I20260808 23:29:49.370604 29174 heartbeater.cc:511] Master 127.9.44.126:39711 requested a full tablet report, sending...
I20260808 23:29:49.373212 28867 ts_manager.cc:194] Registered new tserver with Master: b665a9d8e0534eb2a5a48ba61d21b5d8 (127.9.44.65:42727)
I20260808 23:29:49.375293 28867 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.65@KRBTEST.COM'} at 127.9.44.65:48533
WARNING: no policy specified for kudu/127.9.44.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.66@KRBTEST.COM" created.
Aug 08 23:29:49 dist-test-slave-6bbx krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231788, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for HTTP/127.9.44.84@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
I20260808 23:29:49.421175 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.44.84:34085/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:0
--local_ip_for_outbound_sockets=127.9.44.66
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:39711
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:36417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:29:49 dist-test-slave-6bbx krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231788, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for HTTP/127.9.44.84@KRBTEST.COM
W20260808 23:29:49.970329 29183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:29:49.970575 29183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:29:49.970698 29183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:29:49.977556 29183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:29:49.977648 29183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:29:49.977808 29183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:29:49.986929 29183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:36417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.44.84:34085/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:39711
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:29:49.988095 29183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:29:49.989285 29183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:29:50.001397 29206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:29:50.002032 29209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:29:50.002278 29212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:29:50.002729 29183 server_base.cc:1034] running on GCE node
I20260808 23:29:50.003298 29183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:29:50.004282 29183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:29:50.005494 29183 hybrid_clock.cc:648] HybridClock initialized: now 1786231790005448 us; error 40 us; skew 500 ppm
I20260808 23:29:50.005791 29183 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:29:50 dist-test-slave-6bbx krb5kdc[28140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231790, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:29:50.009045 29183 init.cc:377] Logged in from keytab as kudu/127.9.44.66@KRBTEST.COM (short username kudu)
I20260808 23:29:50.010777 29183 webserver.cc:460] Webserver started at http://127.9.44.66:39793/ using document root <none> and password file <none>
I20260808 23:29:50.011690 29183 fs_manager.cc:359] Metadata directory not provided
I20260808 23:29:50.011837 29183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:29:50.012184 29183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 23:29:50 dist-test-slave-6bbx krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231790, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for HTTP/127.9.44.84@KRBTEST.COM
I20260808 23:29:50.029911 29183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/data/instance:
uuid: "de2840238c5648a78305cee430a8631b"
format_stamp: "Formatted at 2026-08-08 23:29:50 on dist-test-slave-6bbx"
server_key: "593620cdc221f72f598b6da1c81be06e"
server_key_iv: "cb338497f38454406dd6cf9832fc8423"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:50.030934 29183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance:
uuid: "de2840238c5648a78305cee430a8631b"
format_stamp: "Formatted at 2026-08-08 23:29:50 on dist-test-slave-6bbx"
server_key: "593620cdc221f72f598b6da1c81be06e"
server_key_iv: "cb338497f38454406dd6cf9832fc8423"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:50.036244 29183 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 23:29:50.052958 29221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:50.054661 29183 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 23:29:50.054893 29183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "de2840238c5648a78305cee430a8631b"
format_stamp: "Formatted at 2026-08-08 23:29:50 on dist-test-slave-6bbx"
server_key: "593620cdc221f72f598b6da1c81be06e"
server_key_iv: "cb338497f38454406dd6cf9832fc8423"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:50.055204 29183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:29:50.075848 29183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:29:50.081287 29183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:29:50.081674 29183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:29:50.082940 29183 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:29:50.084765 29183 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:29:50.084896 29183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260808 23:29:50.085044 29183 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:29:50.085135 29183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:50.142580 29183 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:36529
I20260808 23:29:50.142627 29377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:36529 every 8 connection(s)
I20260808 23:29:50.144430 29183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:29:50.149178 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 29183
I20260808 23:29:50.149358 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance
I20260808 23:29:50.149950 9393 external_mini_cluster.cc:1428] Setting key 731c0ae7e80bdd0573a1478be231ca44
WARNING: no policy specified for kudu/127.9.44.67@KRBTEST.COM; defaulting to no policy
Aug 08 23:29:50 dist-test-slave-6bbx krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231790, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.67@KRBTEST.COM" created.
I20260808 23:29:50.198377 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:50.150436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:44257 (local address 127.9.44.126:39711)
0808 23:29:50.168609 (+ 18173us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:50.168618 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:50.171563 (+ 2945us) server_negotiation.cc:407] Connection header received
0808 23:29:50.172505 (+ 942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:50.172511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:50.172685 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:50.172788 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:50.180222 (+ 7434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:50.181307 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:50.183744 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:50.184345 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:50.184447 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:50.188943 (+ 4496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:50.188949 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:50.188952 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:50.188986 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:50.191800 (+ 2814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:50.195543 (+ 3743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:50.195549 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:50.195553 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:50.195737 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:50.196300 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:50.196305 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:50.196308 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:50.196724 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:50.196853 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:50.197797 (+ 944us) server_negotiation.cc:299] Negotiation successful
0808 23:29:50.198028 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18005,"thread_start_us":18175,"threads_started":1}
I20260808 23:29:50.200589 29381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:50.151015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:39711 (local address 127.9.44.66:44257)
0808 23:29:50.169369 (+ 18354us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:50.169444 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:50.172214 (+ 2770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:50.178337 (+ 6123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:50.178350 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:50.179173 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:29:50.180019 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:50.180045 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:50.181504 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:50.181508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:50.183524 (+ 2016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:50.183541 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:50.184848 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:50.184855 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:50.185162 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:50.185969 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:29:50.186002 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:29:50.188713 (+ 2711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:29:50.191926 (+ 3213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:50.191935 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:50.191951 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:50.192508 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:50.195867 (+ 3359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:50.195876 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:50.195880 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:50.196169 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:50.196941 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:29:50.196948 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:29:50.197358 (+ 410us) client_negotiation.cc:769] Sending connection context
0808 23:29:50.198717 (+ 1359us) client_negotiation.cc:240] Negotiation successful
0808 23:29:50.199224 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17981,"thread_start_us":17736,"threads_started":1}
I20260808 23:29:50.202222 29378 heartbeater.cc:347] Connected to a master server at 127.9.44.126:39711
I20260808 23:29:50.203881 29378 heartbeater.cc:464] Registering TS with master...
I20260808 23:29:50.204737 29378 heartbeater.cc:511] Master 127.9.44.126:39711 requested a full tablet report, sending...
I20260808 23:29:50.206336 28867 ts_manager.cc:194] Registered new tserver with Master: de2840238c5648a78305cee430a8631b (127.9.44.66:36529)
I20260808 23:29:50.207399 28867 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.66@KRBTEST.COM'} at 127.9.44.66:44257
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
I20260808 23:29:50.225036 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.44.84:34085/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:0
--local_ip_for_outbound_sockets=127.9.44.67
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:39711
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:36417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:29:50.379030 29174 heartbeater.cc:503] Master 127.9.44.126:39711 was elected leader, sending a full tablet report...
W20260808 23:29:50.548367 29385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:29:50.548635 29385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:29:50.548768 29385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:29:50.555647 29385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:29:50.555773 29385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:29:50.555886 29385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:29:50.565063 29385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:36417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.44.84:34085/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:39711
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:29:50.566109 29385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:29:50.567250 29385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:29:50.579455 29392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:29:50.579488 29396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:29:50.579514 29394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:29:50.580441 29385 server_base.cc:1034] running on GCE node
I20260808 23:29:50.580992 29385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:29:50.581827 29385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:29:50.583017 29385 hybrid_clock.cc:648] HybridClock initialized: now 1786231790582971 us; error 45 us; skew 500 ppm
I20260808 23:29:50.583309 29385 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:29:50 dist-test-slave-6bbx krb5kdc[28140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231790, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:29:50.586303 29385 init.cc:377] Logged in from keytab as kudu/127.9.44.67@KRBTEST.COM (short username kudu)
I20260808 23:29:50.587643 29385 webserver.cc:460] Webserver started at http://127.9.44.67:45163/ using document root <none> and password file <none>
I20260808 23:29:50.588294 29385 fs_manager.cc:359] Metadata directory not provided
I20260808 23:29:50.588397 29385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:29:50.588654 29385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 23:29:50 dist-test-slave-6bbx krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231790, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for HTTP/127.9.44.84@KRBTEST.COM
I20260808 23:29:50.603770 29385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/data/instance:
uuid: "9c1e3fb3cdfb442584aa6beb25ce9a83"
format_stamp: "Formatted at 2026-08-08 23:29:50 on dist-test-slave-6bbx"
server_key: "6e77c1c0cbb70fff4b04dd7f9a8c7bed"
server_key_iv: "b09cdc47780b7ab1a705396ae397b60c"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:50.604771 29385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance:
uuid: "9c1e3fb3cdfb442584aa6beb25ce9a83"
format_stamp: "Formatted at 2026-08-08 23:29:50 on dist-test-slave-6bbx"
server_key: "6e77c1c0cbb70fff4b04dd7f9a8c7bed"
server_key_iv: "b09cdc47780b7ab1a705396ae397b60c"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:50.610416 29385 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260808 23:29:50.625276 29406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:50.626896 29385 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:29:50.627101 29385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "9c1e3fb3cdfb442584aa6beb25ce9a83"
format_stamp: "Formatted at 2026-08-08 23:29:50 on dist-test-slave-6bbx"
server_key: "6e77c1c0cbb70fff4b04dd7f9a8c7bed"
server_key_iv: "b09cdc47780b7ab1a705396ae397b60c"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:50.627338 29385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:29:50.645162 29385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:29:50.648578 29385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:29:50.648852 29385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:29:50.649762 29385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:29:50.651027 29385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:29:50.651099 29385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:50.651192 29385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:29:50.651235 29385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:50.680692 29385 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:38697
I20260808 23:29:50.680733 29569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:38697 every 8 connection(s)
I20260808 23:29:50.681910 29385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:29:50.688491 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 29385
I20260808 23:29:50.688643 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance
I20260808 23:29:50.689108 9393 external_mini_cluster.cc:1428] Setting key 445debeae19d25d5612ef755b0a651c7
Aug 08 23:29:50 dist-test-slave-6bbx krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231790, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:29:50.700363 29380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:50.684273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:48719 (local address 127.9.44.126:39711)
0808 23:29:50.684454 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:50.684459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:50.685617 (+ 1158us) server_negotiation.cc:407] Connection header received
0808 23:29:50.687300 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:50.687304 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:50.687412 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:50.687506 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:50.688922 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:50.689732 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:50.691723 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:50.692295 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:50.692403 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:50.695442 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:50.695447 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:50.695451 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:50.695483 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:50.697850 (+ 2367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:50.698523 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:50.698528 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:50.698530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:50.698660 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:50.698988 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:50.698991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:50.698993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:50.699248 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:50.699354 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:50.699946 (+ 592us) server_negotiation.cc:299] Negotiation successful
0808 23:29:50.700115 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 23:29:50.701160 29572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:50.684722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:39711 (local address 127.9.44.67:48719)
0808 23:29:50.685454 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:50.685491 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:50.687066 (+ 1575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:50.687725 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:50.687733 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:50.688145 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:29:50.688748 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:50.688770 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:50.689922 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:50.689927 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:50.691562 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:50.691575 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:50.692489 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:50.692494 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:50.692681 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:50.693277 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:29:50.693298 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:29:50.695264 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:29:50.697979 (+ 2715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:50.697985 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:50.697996 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:50.698423 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:50.698740 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:50.698742 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:50.698745 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:50.698915 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:50.699332 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:29:50.699337 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:29:50.699592 (+ 255us) client_negotiation.cc:769] Sending connection context
0808 23:29:50.699848 (+ 256us) client_negotiation.cc:240] Negotiation successful
0808 23:29:50.700195 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":288,"threads_started":1}
I20260808 23:29:50.702386 29570 heartbeater.cc:347] Connected to a master server at 127.9.44.126:39711
I20260808 23:29:50.702673 29570 heartbeater.cc:464] Registering TS with master...
I20260808 23:29:50.703490 29570 heartbeater.cc:511] Master 127.9.44.126:39711 requested a full tablet report, sending...
I20260808 23:29:50.704898 28867 ts_manager.cc:194] Registered new tserver with Master: 9c1e3fb3cdfb442584aa6beb25ce9a83 (127.9.44.67:38697)
I20260808 23:29:50.705657 28867 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.67@KRBTEST.COM'} at 127.9.44.67:48719
I20260808 23:29:50.715982 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:29:50.717617 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 28989
Aug 08 23:29:50 dist-test-slave-6bbx krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.65@KRBTEST.COM
I20260808 23:29:50.732703 29576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:50.718474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45030 (local address 127.9.44.65:42727)
0808 23:29:50.719053 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:50.719059 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:50.719100 (+ 41us) server_negotiation.cc:407] Connection header received
0808 23:29:50.719153 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:50.719159 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:50.719325 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:50.719441 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:50.720824 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:50.721815 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:50.723201 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:50.723736 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:50.723833 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:50.727525 (+ 3692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:50.727537 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:50.727549 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:50.727584 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:50.730050 (+ 2466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:50.730667 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:50.730673 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:50.730675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:50.730826 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:50.731187 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:50.731190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:50.731192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:50.731506 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:50.731815 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:50.732133 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 23:29:50.732384 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":279,"threads_started":1}
I20260808 23:29:50.733821 29168 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:29:50.814750 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 28989
I20260808 23:29:50.834884 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 29183
Aug 08 23:29:50 dist-test-slave-6bbx krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.66@KRBTEST.COM
I20260808 23:29:50.850729 29578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:50.835782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52642 (local address 127.9.44.66:36529)
0808 23:29:50.836344 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:50.836351 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:50.836377 (+ 26us) server_negotiation.cc:407] Connection header received
0808 23:29:50.836439 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:50.836446 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:50.836657 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:50.836784 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:50.837955 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:50.839035 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:50.840608 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:50.841151 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:50.841254 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:50.845405 (+ 4151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:50.845416 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:50.845431 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:50.845471 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:50.848322 (+ 2851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:50.848937 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:50.848943 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:50.848946 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:50.849110 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:50.849418 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:50.849421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:50.849423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:50.849742 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:50.849970 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:50.850144 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 23:29:50.850361 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":294,"threads_started":1}
I20260808 23:29:50.852005 29373 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:29:50.931062 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 29183
I20260808 23:29:50.954973 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 29385
Aug 08 23:29:50 dist-test-slave-6bbx krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.67@KRBTEST.COM
I20260808 23:29:50.970248 29580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:50.955997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36180 (local address 127.9.44.67:38697)
0808 23:29:50.956633 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:50.956640 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:50.956668 (+ 28us) server_negotiation.cc:407] Connection header received
0808 23:29:50.956741 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:50.956747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:50.956961 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:50.957085 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:50.958333 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:50.959362 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:50.960673 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:50.961239 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:50.961330 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:50.965348 (+ 4018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:50.965359 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:50.965373 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:50.965404 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:50.967880 (+ 2476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:50.968469 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:50.968475 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:50.968478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:50.968597 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:50.968941 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:50.968944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:50.968946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:50.969254 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:50.969489 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:50.969701 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 23:29:50.969910 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":340,"threads_started":1}
I20260808 23:29:50.971503 29563 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:29:51.051872 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 29385
I20260808 23:29:51.070097 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 28819
I20260808 23:29:51.071269 28914 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:29:51.177105 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 28819
I20260808 23:29:51.212862 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:39711
--webserver_interface=127.9.44.126
--webserver_port=45243
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:36417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:39711
--ranger_config_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.44.84:34085/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.44.84:34085/kms with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:29:51.536206 29584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:29:51.536473 29584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:29:51.536567 29584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:29:51.543972 29584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:29:51.544070 29584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:29:51.544123 29584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:29:51.544176 29584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:29:51.544221 29584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:29:51.554733 29584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:36417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.44.84:34085/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:39711
--ranger_config_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:39711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=45243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:29:51.555912 29584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:29:51.557156 29584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:29:51.568408 29592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:29:51.568442 29590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:29:51.568378 29596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:29:51.569322 29584 server_base.cc:1034] running on GCE node
I20260808 23:29:51.570009 29584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:29:51.571228 29584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:29:51.572417 29584 hybrid_clock.cc:648] HybridClock initialized: now 1786231791572378 us; error 29 us; skew 500 ppm
I20260808 23:29:51.572747 29584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:29:51 dist-test-slave-6bbx krb5kdc[28140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231791, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:29:51.575773 29584 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:29:51.577121 29584 webserver.cc:460] Webserver started at http://127.9.44.126:45243/ using document root <none> and password file <none>
I20260808 23:29:51.577737 29584 fs_manager.cc:359] Metadata directory not provided
I20260808 23:29:51.577867 29584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 23:29:51 dist-test-slave-6bbx krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231791, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for HTTP/127.9.44.84@KRBTEST.COM
I20260808 23:29:51.654018 29584 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 23:29:51.657012 29605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:51.658315 29584 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:29:51.658504 29584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "bdc6add1d3f14011aea938feb0d08f39"
format_stamp: "Formatted at 2026-08-08 23:29:47 on dist-test-slave-6bbx"
server_key: "16d1e8e58baa7b267f719cb2bba061f8"
server_key_iv: "bb12b3743a4b8a72426a9c59982484e0"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:51.659443 29584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:29:51.679078 29584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:29:51.682338 29584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:29:51.682595 29584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:29:51.699304 29584 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:39711
I20260808 23:29:51.699323 29677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:39711 every 8 connection(s)
I20260808 23:29:51.700819 29584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:29:51.707351 29678 sys_catalog.cc:263] Verifying existing consensus state
I20260808 23:29:51.707819 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 29584
I20260808 23:29:51.708745 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.44.84:34085/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:42727
--local_ip_for_outbound_sockets=127.9.44.65
--tserver_master_addrs=127.9.44.126:39711
--webserver_port=35931
--webserver_interface=127.9.44.65
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:36417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.44.84:34085/kms with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:29:51.710841 29678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39: Bootstrap starting.
I20260808 23:29:51.731809 29678 log.cc:824] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39: Log is configured to *not* fsync() on all Append() calls
I20260808 23:29:51.743641 29678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:29:51.744251 29678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39: Bootstrap complete.
I20260808 23:29:51.757495 29678 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc6add1d3f14011aea938feb0d08f39" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39711 } }
I20260808 23:29:51.758646 29678 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdc6add1d3f14011aea938feb0d08f39, State: Initialized, Role: FOLLOWER
I20260808 23:29:51.759341 29678 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc6add1d3f14011aea938feb0d08f39" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39711 } }
I20260808 23:29:51.759613 29678 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:29:51.759757 29678 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:29:51.759941 29678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [term 1 FOLLOWER]: Advancing to term 2
I20260808 23:29:51.765420 29678 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc6add1d3f14011aea938feb0d08f39" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39711 } }
I20260808 23:29:51.765983 29678 leader_election.cc:302] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdc6add1d3f14011aea938feb0d08f39; no voters:
I20260808 23:29:51.766578 29678 leader_election.cc:288] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 23:29:51.766692 29687 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 23:29:51.767693 29687 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [term 2 LEADER]: Becoming Leader. State: Replica: bdc6add1d3f14011aea938feb0d08f39, State: Running, Role: LEADER
I20260808 23:29:51.768200 29687 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc6add1d3f14011aea938feb0d08f39" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39711 } }
I20260808 23:29:51.768622 29678 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:29:51.770530 29689 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdc6add1d3f14011aea938feb0d08f39. Latest consensus state: current_term: 2 leader_uuid: "bdc6add1d3f14011aea938feb0d08f39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc6add1d3f14011aea938feb0d08f39" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39711 } } }
I20260808 23:29:51.770524 29688 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bdc6add1d3f14011aea938feb0d08f39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc6add1d3f14011aea938feb0d08f39" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39711 } } }
I20260808 23:29:51.770938 29689 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [sys.catalog]: This master's current role is: LEADER
I20260808 23:29:51.770936 29688 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39 [sys.catalog]: This master's current role is: LEADER
I20260808 23:29:51.775431 29693 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:29:51.781220 29693 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:29:51.782785 29693 catalog_manager.cc:1259] Loaded cluster ID: 62eb6a53ff8d4ecda5c2e1400b43aa41
I20260808 23:29:51.782953 29693 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:29:51.787118 29693 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:29:51.788872 29693 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39: Loaded TSK: 0
I20260808 23:29:51.790449 29693 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:29:51.795133 29678 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260808 23:29:52.179636 29681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:29:52.180154 29681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:29:52.180377 29681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:29:52.191516 29681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:29:52.191653 29681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:29:52.191880 29681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:29:52.207015 29681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:36417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.44.84:34085/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:42727
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=35931
--tserver_master_addrs=127.9.44.126:39711
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:29:52.208817 29681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:29:52.210572 29681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:29:52.231838 29731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:29:52.232810 29730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:29:52.233299 29681 server_base.cc:1034] running on GCE node
W20260808 23:29:52.233366 29733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:29:52.234141 29681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:29:52.235361 29681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:29:52.236644 29681 hybrid_clock.cc:648] HybridClock initialized: now 1786231792236575 us; error 53 us; skew 500 ppm
I20260808 23:29:52.237033 29681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:29:52 dist-test-slave-6bbx krb5kdc[28140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231792, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:29:52.241250 29681 init.cc:377] Logged in from keytab as kudu/127.9.44.65@KRBTEST.COM (short username kudu)
I20260808 23:29:52.245028 29681 webserver.cc:460] Webserver started at http://127.9.44.65:35931/ using document root <none> and password file <none>
I20260808 23:29:52.245950 29681 fs_manager.cc:359] Metadata directory not provided
I20260808 23:29:52.246114 29681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 23:29:52 dist-test-slave-6bbx krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231792, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for HTTP/127.9.44.84@KRBTEST.COM
I20260808 23:29:52.274519 29681 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 23:29:52.278045 29744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:52.279735 29681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:29:52.279958 29681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "b665a9d8e0534eb2a5a48ba61d21b5d8"
format_stamp: "Formatted at 2026-08-08 23:29:49 on dist-test-slave-6bbx"
server_key: "aa51e40c9edbf678468623e4ee3ca2f9"
server_key_iv: "4ab87462fe57892c02d5d01c8840ebcd"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:52.281461 29681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:29:52.311837 29681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:29:52.316988 29681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:29:52.317363 29681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:29:52.318645 29681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:29:52.320499 29681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:29:52.320662 29681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260808 23:29:52.320839 29681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:29:52.320961 29681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:52.393575 29681 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:42727
I20260808 23:29:52.393661 29921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:42727 every 8 connection(s)
I20260808 23:29:52.395381 29681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:29:52.405606 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 29681
I20260808 23:29:52.406412 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.44.84:34085/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:36529
--local_ip_for_outbound_sockets=127.9.44.66
--tserver_master_addrs=127.9.44.126:39711
--webserver_port=39793
--webserver_interface=127.9.44.66
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:36417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.44.84:34085/kms with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:29:52 dist-test-slave-6bbx krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231792, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:29:52.428635 29927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:52.398490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:39711 (local address 127.9.44.65:49849)
0808 23:29:52.403250 (+ 4760us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:52.403315 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:52.405761 (+ 2446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:52.408732 (+ 2971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:52.408744 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:52.409314 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:29:52.410117 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:52.410144 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:52.412556 (+ 2412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:52.412562 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:52.414261 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:52.414277 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:52.415576 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:52.415584 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:52.415911 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:52.416718 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:29:52.416751 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:29:52.419495 (+ 2744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:29:52.423863 (+ 4368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:52.423872 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:52.423886 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:52.424441 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:52.424999 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:52.425003 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:52.425007 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:52.425241 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:52.426183 (+ 942us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:29:52.426191 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:29:52.426564 (+ 373us) client_negotiation.cc:769] Sending connection context
0808 23:29:52.426841 (+ 277us) client_negotiation.cc:240] Negotiation successful
0808 23:29:52.427339 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4456,"thread_start_us":4269,"threads_started":1}
I20260808 23:29:52.431854 29926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:52.398408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:49849 (local address 127.9.44.126:39711)
0808 23:29:52.406555 (+ 8147us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:52.406573 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:52.406621 (+ 48us) server_negotiation.cc:407] Connection header received
0808 23:29:52.407050 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:52.407069 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:52.407534 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:52.408002 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:52.410375 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:52.412375 (+ 2000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:52.414443 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:52.415341 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:52.415514 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:52.419836 (+ 4322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:52.419845 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:52.419863 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:52.419910 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:52.423660 (+ 3750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:52.424634 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:52.424643 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:52.424652 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:52.424858 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:52.425396 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:52.425401 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:52.425405 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:52.426033 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:52.430096 (+ 4063us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:52.430338 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 23:29:52.430728 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7880,"thread_start_us":7648,"threads_started":1}
I20260808 23:29:52.435436 29923 heartbeater.cc:347] Connected to a master server at 127.9.44.126:39711
I20260808 23:29:52.435809 29923 heartbeater.cc:464] Registering TS with master...
I20260808 23:29:52.436723 29923 heartbeater.cc:511] Master 127.9.44.126:39711 requested a full tablet report, sending...
W20260808 23:29:52.440029 29923 heartbeater.cc:644] Failed to heartbeat to 127.9.44.126:39711 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 23:29:52.440248 29923 heartbeater.cc:464] Registering TS with master...
I20260808 23:29:52.440505 29923 heartbeater.cc:511] Master 127.9.44.126:39711 requested a full tablet report, sending...
I20260808 23:29:52.441751 29923 heartbeater.cc:464] Registering TS with master...
I20260808 23:29:52.442092 29923 heartbeater.cc:511] Master 127.9.44.126:39711 requested a full tablet report, sending...
W20260808 23:29:52.443267 29923 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 23:29:52.786362 29928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:29:52.786607 29928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:29:52.786721 29928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:29:52.797008 29928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:29:52.797108 29928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:29:52.797251 29928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:29:52.807900 29928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:36417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.44.84:34085/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:36529
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=39793
--tserver_master_addrs=127.9.44.126:39711
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:29:52.809080 29928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:29:52.810349 29928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:29:52.825467 29957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:29:52.826092 29955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:29:52.826880 29961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:29:52.828315 29928 server_base.cc:1034] running on GCE node
I20260808 23:29:52.828919 29928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:29:52.829878 29928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:29:52.831156 29928 hybrid_clock.cc:648] HybridClock initialized: now 1786231792831100 us; error 45 us; skew 500 ppm
I20260808 23:29:52.831461 29928 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:29:52 dist-test-slave-6bbx krb5kdc[28140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231792, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:29:52.839357 29928 init.cc:377] Logged in from keytab as kudu/127.9.44.66@KRBTEST.COM (short username kudu)
I20260808 23:29:52.841220 29928 webserver.cc:460] Webserver started at http://127.9.44.66:39793/ using document root <none> and password file <none>
I20260808 23:29:52.842064 29928 fs_manager.cc:359] Metadata directory not provided
I20260808 23:29:52.842227 29928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 23:29:52 dist-test-slave-6bbx krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231792, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for HTTP/127.9.44.84@KRBTEST.COM
I20260808 23:29:52.867034 29928 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 23:29:52.870114 29976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:52.871433 29928 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:29:52.871577 29928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "de2840238c5648a78305cee430a8631b"
format_stamp: "Formatted at 2026-08-08 23:29:50 on dist-test-slave-6bbx"
server_key: "593620cdc221f72f598b6da1c81be06e"
server_key_iv: "cb338497f38454406dd6cf9832fc8423"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:52.872562 29928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:29:52.895398 29928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:29:52.900293 29928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:29:52.900630 29928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:29:52.911409 29928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:29:52.913206 29928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:29:52.913334 29928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:52.913488 29928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:29:52.913586 29928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:52.983197 29928 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:36529
I20260808 23:29:52.983276 30157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:36529 every 8 connection(s)
I20260808 23:29:52.985359 29928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:29:52.990087 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 29928
I20260808 23:29:52.990867 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.44.84:34085/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:38697
--local_ip_for_outbound_sockets=127.9.44.67
--tserver_master_addrs=127.9.44.126:39711
--webserver_port=45163
--webserver_interface=127.9.44.67
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:36417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.44.84:34085/kms with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:29:52 dist-test-slave-6bbx krb5kdc[28140](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786231792, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 23:29:53 dist-test-slave-6bbx krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231792, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:29:53.044132 30164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:52.989279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:39711 (local address 127.9.44.66:34287)
0808 23:29:53.004213 (+ 14934us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:53.004269 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:53.006315 (+ 2046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:53.012742 (+ 6427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:53.012757 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:53.013384 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:29:53.014272 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:53.014301 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:53.016084 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:53.016090 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:53.018184 (+ 2094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:53.018199 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:53.019324 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:53.019332 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:53.019623 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:53.020543 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:29:53.020575 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:29:53.026084 (+ 5509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:29:53.032404 (+ 6320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:53.032414 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:53.032431 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:53.033014 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:53.040291 (+ 7277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:53.040296 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:53.040300 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:53.040558 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:53.041304 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:29:53.041313 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:29:53.041747 (+ 434us) client_negotiation.cc:769] Sending connection context
0808 23:29:53.042065 (+ 318us) client_negotiation.cc:240] Negotiation successful
0808 23:29:53.042679 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14630,"thread_start_us":14399,"threads_started":1}
I20260808 23:29:53.045665 30162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:52.988602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:34287 (local address 127.9.44.126:39711)
0808 23:29:53.012073 (+ 23471us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:53.012080 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:53.012104 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:29:53.012188 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:53.012194 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:53.012280 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:53.014483 (+ 2203us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:53.014688 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:53.015877 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:53.018379 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:53.019050 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:53.021947 (+ 2897us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:53.028258 (+ 6311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:53.028265 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:53.028270 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:53.028310 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:53.032248 (+ 3938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:53.039963 (+ 7715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:53.039969 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:53.039972 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:53.040144 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:53.040740 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:53.040744 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:53.040747 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:53.041136 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:53.044944 (+ 3808us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:53.045083 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 23:29:53.045301 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23308,"thread_start_us":23191,"threads_started":1}
I20260808 23:29:53.046837 30158 heartbeater.cc:347] Connected to a master server at 127.9.44.126:39711
I20260808 23:29:53.047173 30158 heartbeater.cc:464] Registering TS with master...
I20260808 23:29:53.048107 30158 heartbeater.cc:511] Master 127.9.44.126:39711 requested a full tablet report, sending...
I20260808 23:29:53.050120 29631 ts_manager.cc:194] Registered new tserver with Master: de2840238c5648a78305cee430a8631b (127.9.44.66:36529)
I20260808 23:29:53.052325 29631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.66@KRBTEST.COM'} at 127.9.44.66:34287
W20260808 23:29:53.408337 30165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:29:53.408601 30165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:29:53.408756 30165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:29:53.416133 30165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:29:53.416324 30165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:29:53.416487 30165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:29:53.428686 30165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:36417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.44.84:34085/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:38697
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=45163
--tserver_master_addrs=127.9.44.126:39711
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:29:53.430235 30165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:29:53.431958 30165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 23:29:53.443586 29923 heartbeater.cc:464] Registering TS with master...
I20260808 23:29:53.444141 29923 heartbeater.cc:511] Master 127.9.44.126:39711 requested a full tablet report, sending...
I20260808 23:29:53.445537 29631 ts_manager.cc:194] Registered new tserver with Master: b665a9d8e0534eb2a5a48ba61d21b5d8 (127.9.44.65:42727)
I20260808 23:29:53.446825 29631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.65@KRBTEST.COM'} at 127.9.44.65:49849
Aug 08 23:29:53 dist-test-slave-6bbx krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231792, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for HTTP/127.9.44.84@KRBTEST.COM
W20260808 23:29:53.458976 30173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:29:53.458966 30172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:29:53.468618 30178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:29:53.472465 30165 server_base.cc:1034] running on GCE node
I20260808 23:29:53.473183 30165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:29:53.474852 30165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:29:53.477500 30165 hybrid_clock.cc:648] HybridClock initialized: now 1786231793477422 us; error 58 us; skew 500 ppm
I20260808 23:29:53.477890 30165 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:29:53 dist-test-slave-6bbx krb5kdc[28140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231793, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:29:53.484892 30165 init.cc:377] Logged in from keytab as kudu/127.9.44.67@KRBTEST.COM (short username kudu)
I20260808 23:29:53.543651 30165 webserver.cc:460] Webserver started at http://127.9.44.67:45163/ using document root <none> and password file <none>
I20260808 23:29:53.544782 30165 fs_manager.cc:359] Metadata directory not provided
I20260808 23:29:53.545115 30165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 23:29:53 dist-test-slave-6bbx krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231793, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for HTTP/127.9.44.84@KRBTEST.COM
I20260808 23:29:53.582980 30165 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.004s sys 0.008s
I20260808 23:29:53.586740 30190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:53.588583 30165 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 23:29:53.588775 30165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "9c1e3fb3cdfb442584aa6beb25ce9a83"
format_stamp: "Formatted at 2026-08-08 23:29:50 on dist-test-slave-6bbx"
server_key: "6e77c1c0cbb70fff4b04dd7f9a8c7bed"
server_key_iv: "b09cdc47780b7ab1a705396ae397b60c"
server_key_version: "kuduclusterkey@0"
I20260808 23:29:53.590143 30165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:29:53.626044 30165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:29:53.631064 30165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:29:53.631367 30165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:29:53.632601 30165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:29:53.634339 30165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:29:53.634464 30165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:53.634565 30165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:29:53.634634 30165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:53.716537 30165 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:38697
I20260808 23:29:53.716620 30356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:38697 every 8 connection(s)
I20260808 23:29:53.718533 30165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:29:53.724553 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 30165
Aug 08 23:29:53 dist-test-slave-6bbx krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231792, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for HTTP/127.9.44.84@KRBTEST.COM
I20260808 23:29:53.760385 30360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:53.727412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48426 (local address 127.9.44.126:39711)
0808 23:29:53.743911 (+ 16499us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:53.743917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:53.743936 (+ 19us) server_negotiation.cc:407] Connection header received
0808 23:29:53.743995 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:53.743999 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:53.744083 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:53.744183 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:53.746186 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:53.747273 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:53.749235 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:53.749859 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:53.749968 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:53.753281 (+ 3313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:53.753288 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:53.753292 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:53.753330 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:53.756658 (+ 3328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:53.757458 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:53.757463 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:53.757466 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:53.757615 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:53.758062 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:53.758067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:53.758070 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:53.758408 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:53.758673 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:53.759784 (+ 1111us) server_negotiation.cc:299] Negotiation successful
0808 23:29:53.760003 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16378,"thread_start_us":16271,"threads_started":1}
Aug 08 23:29:53 dist-test-slave-6bbx krb5kdc[28140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231793, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:29:53.782243 30364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:53.727861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:39711 (local address 127.9.44.67:46399)
0808 23:29:53.739189 (+ 11328us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:53.739255 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:53.741737 (+ 2482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:53.756769 (+ 15032us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:53.756783 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:53.757396 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:29:53.758300 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:53.758327 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:53.759989 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:53.759995 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:53.762410 (+ 2415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:53.762426 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:53.764039 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:53.764046 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:53.764323 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:53.764998 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:29:53.765028 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:29:53.772113 (+ 7085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:29:53.777195 (+ 5082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:53.777204 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:53.777221 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:53.778019 (+ 798us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:53.778448 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:53.778453 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:53.778457 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:53.778691 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:53.779437 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:29:53.779444 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:29:53.779923 (+ 479us) client_negotiation.cc:769] Sending connection context
0808 23:29:53.780233 (+ 310us) client_negotiation.cc:240] Negotiation successful
0808 23:29:53.780803 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11010,"thread_start_us":14164,"threads_started":1}
I20260808 23:29:53.783345 30366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:53.743862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:46399 (local address 127.9.44.126:39711)
0808 23:29:53.756104 (+ 12242us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:53.756110 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:53.756142 (+ 32us) server_negotiation.cc:407] Connection header received
0808 23:29:53.756189 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:53.756193 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:53.756274 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:53.758480 (+ 2206us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:53.758713 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:53.759794 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:53.762585 (+ 2791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:53.763271 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:53.763743 (+ 472us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:53.772405 (+ 8662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:53.772412 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:53.772416 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:53.772452 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:53.777024 (+ 4572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:53.778161 (+ 1137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:53.778166 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:53.778170 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:53.778308 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:53.778850 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:53.778855 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:53.778858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:53.779203 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:53.779347 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:53.782782 (+ 3435us) server_negotiation.cc:299] Negotiation successful
0808 23:29:53.782993 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12107,"thread_start_us":11957,"threads_started":1}
I20260808 23:29:53.784744 30358 heartbeater.cc:347] Connected to a master server at 127.9.44.126:39711
I20260808 23:29:53.785063 30358 heartbeater.cc:464] Registering TS with master...
I20260808 23:29:53.785861 30358 heartbeater.cc:511] Master 127.9.44.126:39711 requested a full tablet report, sending...
I20260808 23:29:53.788069 29631 ts_manager.cc:194] Registered new tserver with Master: 9c1e3fb3cdfb442584aa6beb25ce9a83 (127.9.44.67:38697)
I20260808 23:29:53.789104 29631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.67@KRBTEST.COM'} at 127.9.44.67:46399
I20260808 23:29:53.794097 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:29:53.838928 30366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:53.810103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48440 (local address 127.9.44.126:39711)
0808 23:29:53.810267 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:53.810273 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:53.822242 (+ 11969us) server_negotiation.cc:407] Connection header received
0808 23:29:53.822763 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:53.822770 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:53.822864 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:53.822940 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:53.825086 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:53.826223 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:53.828227 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:53.828868 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:53.828963 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:53.832458 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:53.832464 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:53.832469 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:53.832510 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:53.835819 (+ 3309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:53.836606 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:53.836612 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:53.836615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:53.836769 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:53.837187 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:53.837192 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:53.837195 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:53.837542 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:53.837760 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:53.838323 (+ 563us) server_negotiation.cc:299] Negotiation successful
0808 23:29:53.838550 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 23:29:53.845940 29627 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48440:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:29:53.857538 29627 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:29:53.938979 30384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:53.906586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:36529 (local address 127.0.0.1:52648)
0808 23:29:53.912966 (+ 6380us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:53.912984 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:53.913101 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:53.916997 (+ 3896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:53.917005 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:53.917044 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:29:53.917493 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:53.917506 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:53.924562 (+ 7056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:53.924567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:53.927279 (+ 2712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:53.927296 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:53.938034 (+ 10738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:53.938041 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:53.938240 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:53.938301 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 23:29:53.938413 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 23:29:53.938613 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6245,"thread_start_us":6118,"threads_started":1}
I20260808 23:29:53.940594 30385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:53.906879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52648 (local address 127.9.44.66:36529)
0808 23:29:53.916095 (+ 9216us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:53.916105 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:53.916145 (+ 40us) server_negotiation.cc:407] Connection header received
0808 23:29:53.916251 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:53.916260 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:53.916512 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:53.916705 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:29:53.922729 (+ 6024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:53.924400 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:53.934493 (+ 10093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:53.936029 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:53.939821 (+ 3792us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:53.940022 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:53.940097 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 23:29:53.940279 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9050,"thread_start_us":8913,"threads_started":1}
I20260808 23:29:53.941772 30380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:53.904112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:38697 (local address 127.0.0.1:36184)
0808 23:29:53.911071 (+ 6959us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:53.911119 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:53.911339 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:53.918984 (+ 7645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:53.918989 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:53.919017 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:29:53.919396 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:53.919406 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:53.932237 (+ 12831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:53.932243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:53.934335 (+ 2092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:53.934348 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:53.941003 (+ 6655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:53.941011 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:53.941174 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:53.941208 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 23:29:53.941278 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 23:29:53.941403 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6652,"mutex_wait_us":27,"thread_start_us":6502,"threads_started":1}
I20260808 23:29:53.943854 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:53.904344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36184 (local address 127.9.44.67:38697)
0808 23:29:53.918106 (+ 13762us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:53.918115 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:53.918162 (+ 47us) server_negotiation.cc:407] Connection header received
0808 23:29:53.918260 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:53.918267 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:53.918491 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:53.918671 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:29:53.924611 (+ 5940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:53.926220 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:53.936243 (+ 10023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:53.937765 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:53.942274 (+ 4509us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:53.942503 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:53.942574 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 23:29:53.943460 (+ 886us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13590,"thread_start_us":13431,"threads_started":1}
I20260808 23:29:53.951076 30386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:53.911551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:42727 (local address 127.0.0.1:45042)
0808 23:29:53.922442 (+ 10891us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:53.922463 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:53.922573 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:53.927585 (+ 5012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:53.927590 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:53.927616 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:29:53.928027 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:53.928040 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:53.932699 (+ 4659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:53.932705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:53.936216 (+ 3511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:53.936232 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:53.950322 (+ 14090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:53.950329 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:53.950504 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:53.950537 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 23:29:53.950619 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 23:29:53.950766 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10768,"thread_start_us":10643,"threads_started":1}
I20260808 23:29:53.951900 30388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:53.911926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45042 (local address 127.9.44.65:42727)
0808 23:29:53.926742 (+ 14816us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:53.926750 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:53.926777 (+ 27us) server_negotiation.cc:407] Connection header received
0808 23:29:53.926870 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:53.926877 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:53.927121 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:53.927328 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:29:53.930467 (+ 3139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:53.932531 (+ 2064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:53.947268 (+ 14737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:53.949827 (+ 2559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:53.949955 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:53.950169 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:53.951368 (+ 1199us) server_negotiation.cc:299] Negotiation successful
0808 23:29:53.951534 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14640,"thread_start_us":15730,"threads_started":1}
I20260808 23:29:53.951495 30267 tablet_service.cc:1467] Processing CreateTablet for tablet 9b7c7a47f8624673b9b9626679848b0c (DEFAULT_TABLE table=test-table [id=0e3475bb9dc043ec96fbc61bda23cf95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:29:53.955394 29823 tablet_service.cc:1467] Processing CreateTablet for tablet 9b7c7a47f8624673b9b9626679848b0c (DEFAULT_TABLE table=test-table [id=0e3475bb9dc043ec96fbc61bda23cf95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:29:53.957010 29823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b7c7a47f8624673b9b9626679848b0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:29:53.957285 30267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b7c7a47f8624673b9b9626679848b0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:29:53.962787 30055 tablet_service.cc:1467] Processing CreateTablet for tablet 9b7c7a47f8624673b9b9626679848b0c (DEFAULT_TABLE table=test-table [id=0e3475bb9dc043ec96fbc61bda23cf95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:29:53.965310 30055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b7c7a47f8624673b9b9626679848b0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:29:53.979140 30389 tablet_bootstrap.cc:492] T 9b7c7a47f8624673b9b9626679848b0c P b665a9d8e0534eb2a5a48ba61d21b5d8: Bootstrap starting.
I20260808 23:29:53.997329 30389 tablet_bootstrap.cc:654] T 9b7c7a47f8624673b9b9626679848b0c P b665a9d8e0534eb2a5a48ba61d21b5d8: Neither blocks nor log segments found. Creating new log.
I20260808 23:29:53.999312 30389 log.cc:824] T 9b7c7a47f8624673b9b9626679848b0c P b665a9d8e0534eb2a5a48ba61d21b5d8: Log is configured to *not* fsync() on all Append() calls
I20260808 23:29:54.005831 30393 tablet_bootstrap.cc:492] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83: Bootstrap starting.
I20260808 23:29:54.010445 30393 tablet_bootstrap.cc:654] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83: Neither blocks nor log segments found. Creating new log.
I20260808 23:29:54.012914 30395 tablet_bootstrap.cc:492] T 9b7c7a47f8624673b9b9626679848b0c P de2840238c5648a78305cee430a8631b: Bootstrap starting.
I20260808 23:29:54.015533 30389 tablet_bootstrap.cc:492] T 9b7c7a47f8624673b9b9626679848b0c P b665a9d8e0534eb2a5a48ba61d21b5d8: No bootstrap required, opened a new log
I20260808 23:29:54.015928 30389 ts_tablet_manager.cc:1397] T 9b7c7a47f8624673b9b9626679848b0c P b665a9d8e0534eb2a5a48ba61d21b5d8: Time spent bootstrapping tablet: real 0.037s user 0.010s sys 0.004s
I20260808 23:29:54.026961 30395 tablet_bootstrap.cc:654] T 9b7c7a47f8624673b9b9626679848b0c P de2840238c5648a78305cee430a8631b: Neither blocks nor log segments found. Creating new log.
I20260808 23:29:54.028475 30393 log.cc:824] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83: Log is configured to *not* fsync() on all Append() calls
I20260808 23:29:54.030309 30395 log.cc:824] T 9b7c7a47f8624673b9b9626679848b0c P de2840238c5648a78305cee430a8631b: Log is configured to *not* fsync() on all Append() calls
I20260808 23:29:54.034828 30389 raft_consensus.cc:348] T 9b7c7a47f8624673b9b9626679848b0c P b665a9d8e0534eb2a5a48ba61d21b5d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1e3fb3cdfb442584aa6beb25ce9a83" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38697 } } peers { permanent_uuid: "b665a9d8e0534eb2a5a48ba61d21b5d8" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42727 } } peers { permanent_uuid: "de2840238c5648a78305cee430a8631b" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 36529 } }
I20260808 23:29:54.035202 30389 raft_consensus.cc:374] T 9b7c7a47f8624673b9b9626679848b0c P b665a9d8e0534eb2a5a48ba61d21b5d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:29:54.035315 30389 raft_consensus.cc:729] T 9b7c7a47f8624673b9b9626679848b0c P b665a9d8e0534eb2a5a48ba61d21b5d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b665a9d8e0534eb2a5a48ba61d21b5d8, State: Initialized, Role: FOLLOWER
I20260808 23:29:54.038352 30395 tablet_bootstrap.cc:492] T 9b7c7a47f8624673b9b9626679848b0c P de2840238c5648a78305cee430a8631b: No bootstrap required, opened a new log
I20260808 23:29:54.038607 30395 ts_tablet_manager.cc:1397] T 9b7c7a47f8624673b9b9626679848b0c P de2840238c5648a78305cee430a8631b: Time spent bootstrapping tablet: real 0.026s user 0.006s sys 0.000s
I20260808 23:29:54.043081 30393 tablet_bootstrap.cc:492] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83: No bootstrap required, opened a new log
I20260808 23:29:54.043404 30393 ts_tablet_manager.cc:1397] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83: Time spent bootstrapping tablet: real 0.038s user 0.009s sys 0.000s
I20260808 23:29:54.048034 30389 consensus_queue.cc:260] T 9b7c7a47f8624673b9b9626679848b0c P b665a9d8e0534eb2a5a48ba61d21b5d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1e3fb3cdfb442584aa6beb25ce9a83" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38697 } } peers { permanent_uuid: "b665a9d8e0534eb2a5a48ba61d21b5d8" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42727 } } peers { permanent_uuid: "de2840238c5648a78305cee430a8631b" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 36529 } }
I20260808 23:29:54.049857 30395 raft_consensus.cc:348] T 9b7c7a47f8624673b9b9626679848b0c P de2840238c5648a78305cee430a8631b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1e3fb3cdfb442584aa6beb25ce9a83" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38697 } } peers { permanent_uuid: "b665a9d8e0534eb2a5a48ba61d21b5d8" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42727 } } peers { permanent_uuid: "de2840238c5648a78305cee430a8631b" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 36529 } }
I20260808 23:29:54.050338 30395 raft_consensus.cc:374] T 9b7c7a47f8624673b9b9626679848b0c P de2840238c5648a78305cee430a8631b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:29:54.050469 30395 raft_consensus.cc:729] T 9b7c7a47f8624673b9b9626679848b0c P de2840238c5648a78305cee430a8631b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de2840238c5648a78305cee430a8631b, State: Initialized, Role: FOLLOWER
I20260808 23:29:54.051121 30395 consensus_queue.cc:260] T 9b7c7a47f8624673b9b9626679848b0c P de2840238c5648a78305cee430a8631b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1e3fb3cdfb442584aa6beb25ce9a83" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38697 } } peers { permanent_uuid: "b665a9d8e0534eb2a5a48ba61d21b5d8" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42727 } } peers { permanent_uuid: "de2840238c5648a78305cee430a8631b" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 36529 } }
I20260808 23:29:54.056887 30158 heartbeater.cc:503] Master 127.9.44.126:39711 was elected leader, sending a full tablet report...
I20260808 23:29:54.059106 30389 ts_tablet_manager.cc:1428] T 9b7c7a47f8624673b9b9626679848b0c P b665a9d8e0534eb2a5a48ba61d21b5d8: Time spent starting tablet: real 0.043s user 0.012s sys 0.003s
I20260808 23:29:54.060099 30393 raft_consensus.cc:348] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1e3fb3cdfb442584aa6beb25ce9a83" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38697 } } peers { permanent_uuid: "b665a9d8e0534eb2a5a48ba61d21b5d8" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42727 } } peers { permanent_uuid: "de2840238c5648a78305cee430a8631b" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 36529 } }
I20260808 23:29:54.060709 29923 heartbeater.cc:503] Master 127.9.44.126:39711 was elected leader, sending a full tablet report...
I20260808 23:29:54.062075 30393 raft_consensus.cc:374] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:29:54.062255 30393 raft_consensus.cc:729] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c1e3fb3cdfb442584aa6beb25ce9a83, State: Initialized, Role: FOLLOWER
I20260808 23:29:54.063004 30393 consensus_queue.cc:260] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1e3fb3cdfb442584aa6beb25ce9a83" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38697 } } peers { permanent_uuid: "b665a9d8e0534eb2a5a48ba61d21b5d8" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42727 } } peers { permanent_uuid: "de2840238c5648a78305cee430a8631b" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 36529 } }
I20260808 23:29:54.067909 30395 ts_tablet_manager.cc:1428] T 9b7c7a47f8624673b9b9626679848b0c P de2840238c5648a78305cee430a8631b: Time spent starting tablet: real 0.029s user 0.011s sys 0.008s
I20260808 23:29:54.070937 30358 heartbeater.cc:503] Master 127.9.44.126:39711 was elected leader, sending a full tablet report...
I20260808 23:29:54.071580 30393 ts_tablet_manager.cc:1428] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83: Time spent starting tablet: real 0.028s user 0.018s sys 0.000s
W20260808 23:29:54.148602 29925 tablet.cc:2369] T 9b7c7a47f8624673b9b9626679848b0c P b665a9d8e0534eb2a5a48ba61d21b5d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:29:54.238729 30365 tablet.cc:2369] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:29:54.243772 30160 tablet.cc:2369] T 9b7c7a47f8624673b9b9626679848b0c P de2840238c5648a78305cee430a8631b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:29:54.253573 30404 raft_consensus.cc:482] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:29:54.253844 30404 raft_consensus.cc:504] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1e3fb3cdfb442584aa6beb25ce9a83" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38697 } } peers { permanent_uuid: "b665a9d8e0534eb2a5a48ba61d21b5d8" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42727 } } peers { permanent_uuid: "de2840238c5648a78305cee430a8631b" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 36529 } }
I20260808 23:29:54.255776 30404 leader_election.cc:288] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b665a9d8e0534eb2a5a48ba61d21b5d8 (127.9.44.65:42727), de2840238c5648a78305cee430a8631b (127.9.44.66:36529)
I20260808 23:29:54.261536 30364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:54.256173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:42727 (local address 127.9.44.67:38707)
0808 23:29:54.256486 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:54.256505 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:54.256622 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:54.257057 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:54.257061 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:54.257088 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:29:54.257245 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:54.257253 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:54.258303 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:54.258306 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:54.259587 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:54.259599 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:54.260824 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:54.260830 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:54.261008 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:54.261043 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 23:29:54.261135 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 23:29:54.261263 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213}
I20260808 23:29:54.261579 30388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:54.256217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:38707 (local address 127.9.44.65:42727)
0808 23:29:54.256449 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:54.256457 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:54.256741 (+ 284us) server_negotiation.cc:407] Connection header received
0808 23:29:54.256813 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:54.256816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:54.256886 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:54.256961 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:29:54.257435 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:54.258168 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:54.259779 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:54.260644 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:54.260791 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:54.260889 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:54.261208 (+ 319us) server_negotiation.cc:299] Negotiation successful
0808 23:29:54.261340 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 23:29:54.262576 30385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:54.256258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:43907 (local address 127.9.44.66:36529)
0808 23:29:54.256453 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:54.256459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:54.256973 (+ 514us) server_negotiation.cc:407] Connection header received
0808 23:29:54.257137 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:54.257141 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:54.257212 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:54.257287 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:29:54.257817 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:54.258511 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:54.260387 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:54.261516 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:54.261650 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:54.261737 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:54.262194 (+ 457us) server_negotiation.cc:299] Negotiation successful
0808 23:29:54.262329 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 23:29:54.262651 30413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:54.256212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:36529 (local address 127.9.44.67:43907)
0808 23:29:54.256869 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:54.256883 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:54.256991 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:54.257325 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:54.257329 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:54.257354 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:29:54.257670 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:54.257680 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:54.258659 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:54.258663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:54.260216 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:54.260231 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:54.261852 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:54.261858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:54.262012 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:54.262045 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 23:29:54.262158 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 23:29:54.262281 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":363,"threads_started":1}
I20260808 23:29:54.262912 29851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b7c7a47f8624673b9b9626679848b0c" candidate_uuid: "9c1e3fb3cdfb442584aa6beb25ce9a83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b665a9d8e0534eb2a5a48ba61d21b5d8" is_pre_election: true
I20260808 23:29:54.263538 29851 raft_consensus.cc:2454] T 9b7c7a47f8624673b9b9626679848b0c P b665a9d8e0534eb2a5a48ba61d21b5d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c1e3fb3cdfb442584aa6beb25ce9a83 in term 0.
I20260808 23:29:54.263900 30084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b7c7a47f8624673b9b9626679848b0c" candidate_uuid: "9c1e3fb3cdfb442584aa6beb25ce9a83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de2840238c5648a78305cee430a8631b" is_pre_election: true
I20260808 23:29:54.264272 30192 leader_election.cc:302] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c1e3fb3cdfb442584aa6beb25ce9a83, b665a9d8e0534eb2a5a48ba61d21b5d8; no voters:
I20260808 23:29:54.264439 30084 raft_consensus.cc:2454] T 9b7c7a47f8624673b9b9626679848b0c P de2840238c5648a78305cee430a8631b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c1e3fb3cdfb442584aa6beb25ce9a83 in term 0.
I20260808 23:29:54.264626 30404 raft_consensus.cc:2781] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:29:54.264770 30404 raft_consensus.cc:482] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:29:54.264882 30404 raft_consensus.cc:3037] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:29:54.267024 30404 raft_consensus.cc:504] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1e3fb3cdfb442584aa6beb25ce9a83" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38697 } } peers { permanent_uuid: "b665a9d8e0534eb2a5a48ba61d21b5d8" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42727 } } peers { permanent_uuid: "de2840238c5648a78305cee430a8631b" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 36529 } }
I20260808 23:29:54.267570 30404 leader_election.cc:288] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83 [CANDIDATE]: Term 1 election: Requested vote from peers b665a9d8e0534eb2a5a48ba61d21b5d8 (127.9.44.65:42727), de2840238c5648a78305cee430a8631b (127.9.44.66:36529)
I20260808 23:29:54.267966 29851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b7c7a47f8624673b9b9626679848b0c" candidate_uuid: "9c1e3fb3cdfb442584aa6beb25ce9a83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b665a9d8e0534eb2a5a48ba61d21b5d8"
I20260808 23:29:54.268019 30084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b7c7a47f8624673b9b9626679848b0c" candidate_uuid: "9c1e3fb3cdfb442584aa6beb25ce9a83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de2840238c5648a78305cee430a8631b"
I20260808 23:29:54.268137 29851 raft_consensus.cc:3037] T 9b7c7a47f8624673b9b9626679848b0c P b665a9d8e0534eb2a5a48ba61d21b5d8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:29:54.268167 30084 raft_consensus.cc:3037] T 9b7c7a47f8624673b9b9626679848b0c P de2840238c5648a78305cee430a8631b [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:29:54.270300 29851 raft_consensus.cc:2454] T 9b7c7a47f8624673b9b9626679848b0c P b665a9d8e0534eb2a5a48ba61d21b5d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c1e3fb3cdfb442584aa6beb25ce9a83 in term 1.
I20260808 23:29:54.270385 30084 raft_consensus.cc:2454] T 9b7c7a47f8624673b9b9626679848b0c P de2840238c5648a78305cee430a8631b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c1e3fb3cdfb442584aa6beb25ce9a83 in term 1.
I20260808 23:29:54.270646 30192 leader_election.cc:302] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c1e3fb3cdfb442584aa6beb25ce9a83, b665a9d8e0534eb2a5a48ba61d21b5d8; no voters:
I20260808 23:29:54.270912 30404 raft_consensus.cc:2781] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:29:54.271309 30404 raft_consensus.cc:686] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83 [term 1 LEADER]: Becoming Leader. State: Replica: 9c1e3fb3cdfb442584aa6beb25ce9a83, State: Running, Role: LEADER
I20260808 23:29:54.271771 30404 consensus_queue.cc:237] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1e3fb3cdfb442584aa6beb25ce9a83" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38697 } } peers { permanent_uuid: "b665a9d8e0534eb2a5a48ba61d21b5d8" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42727 } } peers { permanent_uuid: "de2840238c5648a78305cee430a8631b" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 36529 } }
I20260808 23:29:54.275907 29627 catalog_manager.cc:5495] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c1e3fb3cdfb442584aa6beb25ce9a83 (127.9.44.67). New cstate: current_term: 1 leader_uuid: "9c1e3fb3cdfb442584aa6beb25ce9a83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c1e3fb3cdfb442584aa6beb25ce9a83" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b665a9d8e0534eb2a5a48ba61d21b5d8" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de2840238c5648a78305cee430a8631b" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 36529 } health_report { overall_health: UNKNOWN } } }
I20260808 23:29:54.301838 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:54.297028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36190 (local address 127.9.44.67:38697)
0808 23:29:54.297222 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:54.297227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:54.297265 (+ 38us) server_negotiation.cc:407] Connection header received
0808 23:29:54.297530 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:54.297535 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:54.297607 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:54.297722 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:29:54.298055 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:54.298719 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:54.300036 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:54.300431 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:54.300519 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:54.300935 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:29:54.301397 (+ 462us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:29:54.301499 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:54.301600 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 23:29:54.301660 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 23:29:54.309334 29851 raft_consensus.cc:1261] T 9b7c7a47f8624673b9b9626679848b0c P b665a9d8e0534eb2a5a48ba61d21b5d8 [term 1 FOLLOWER]: Refusing update from remote peer 9c1e3fb3cdfb442584aa6beb25ce9a83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:29:54.309338 30084 raft_consensus.cc:1261] T 9b7c7a47f8624673b9b9626679848b0c P de2840238c5648a78305cee430a8631b [term 1 FOLLOWER]: Refusing update from remote peer 9c1e3fb3cdfb442584aa6beb25ce9a83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:29:54.310123 30414 consensus_queue.cc:1035] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b665a9d8e0534eb2a5a48ba61d21b5d8" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:29:54.310456 30404 consensus_queue.cc:1035] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de2840238c5648a78305cee430a8631b" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 36529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:29:54.324108 30421 mvcc.cc:205] Tried to move back new op lower bound from 7316405429482012672 to 7316405429340704768. Current Snapshot: MvccSnapshot[applied={T|T < 7316405429482012672}]
I20260808 23:29:54.338184 29631 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:29:54.338480 29631 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:29:54.342831 29631 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bdc6add1d3f14011aea938feb0d08f39: Sending DeleteTablet for 3 replicas of tablet 9b7c7a47f8624673b9b9626679848b0c
I20260808 23:29:54.344255 30267 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b7c7a47f8624673b9b9626679848b0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:29:54 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:36184
I20260808 23:29:54.344534 29823 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b7c7a47f8624673b9b9626679848b0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:29:54 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:45042
I20260808 23:29:54.345120 30266 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b7c7a47f8624673b9b9626679848b0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:29:54 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:36184
I20260808 23:29:54.345552 30055 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b7c7a47f8624673b9b9626679848b0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:29:54 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:52648
I20260808 23:29:54.346576 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 29681
I20260808 23:29:54.355856 30437 tablet_replica.cc:330] T 9b7c7a47f8624673b9b9626679848b0c P de2840238c5648a78305cee430a8631b: stopping tablet replica
I20260808 23:29:54.356349 30437 raft_consensus.cc:2229] T 9b7c7a47f8624673b9b9626679848b0c P de2840238c5648a78305cee430a8631b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:29:54.356884 30437 raft_consensus.cc:2258] T 9b7c7a47f8624673b9b9626679848b0c P de2840238c5648a78305cee430a8631b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:29:54.359028 30437 ts_tablet_manager.cc:1905] T 9b7c7a47f8624673b9b9626679848b0c P de2840238c5648a78305cee430a8631b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:29:54.360050 30434 tablet_replica.cc:330] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83: stopping tablet replica
I20260808 23:29:54.360421 30434 raft_consensus.cc:2229] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:29:54.361037 30434 raft_consensus.cc:2258] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:29:54.362534 30436 tablet_replica.cc:330] T 9b7c7a47f8624673b9b9626679848b0c P b665a9d8e0534eb2a5a48ba61d21b5d8: stopping tablet replica
I20260808 23:29:54.363914 30434 ts_tablet_manager.cc:1905] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:29:54.364694 30436 raft_consensus.cc:2229] T 9b7c7a47f8624673b9b9626679848b0c P b665a9d8e0534eb2a5a48ba61d21b5d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:29:54.365244 30436 raft_consensus.cc:2258] T 9b7c7a47f8624673b9b9626679848b0c P b665a9d8e0534eb2a5a48ba61d21b5d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:29:54.366066 30437 ts_tablet_manager.cc:1918] T 9b7c7a47f8624673b9b9626679848b0c P de2840238c5648a78305cee430a8631b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:29:54.366272 30437 log.cc:1196] T 9b7c7a47f8624673b9b9626679848b0c P de2840238c5648a78305cee430a8631b: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/9b7c7a47f8624673b9b9626679848b0c
I20260808 23:29:54.366848 30437 ts_tablet_manager.cc:1939] T 9b7c7a47f8624673b9b9626679848b0c P de2840238c5648a78305cee430a8631b: Deleting consensus metadata
I20260808 23:29:54.368634 29606 catalog_manager.cc:4841] TS de2840238c5648a78305cee430a8631b (127.9.44.66:36529): tablet 9b7c7a47f8624673b9b9626679848b0c (table test-table [id=0e3475bb9dc043ec96fbc61bda23cf95]) successfully deleted
I20260808 23:29:54.369884 30436 ts_tablet_manager.cc:1905] T 9b7c7a47f8624673b9b9626679848b0c P b665a9d8e0534eb2a5a48ba61d21b5d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:29:54.376879 30436 ts_tablet_manager.cc:1918] T 9b7c7a47f8624673b9b9626679848b0c P b665a9d8e0534eb2a5a48ba61d21b5d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:29:54.377120 30436 log.cc:1196] T 9b7c7a47f8624673b9b9626679848b0c P b665a9d8e0534eb2a5a48ba61d21b5d8: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-0/wal/wals/9b7c7a47f8624673b9b9626679848b0c
I20260808 23:29:54.377784 30388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:54.348767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45046 (local address 127.9.44.65:42727)
0808 23:29:54.348945 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:54.348951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:54.361893 (+ 12942us) server_negotiation.cc:407] Connection header received
0808 23:29:54.362075 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:54.362080 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:54.362200 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:54.362313 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:54.364348 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:54.365524 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:54.367418 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:54.368092 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:54.368243 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:54.371750 (+ 3507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:54.371761 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:54.371774 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:54.371808 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:54.374593 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:54.375340 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:54.375348 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:54.375351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:54.375488 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:54.375952 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:54.375957 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:54.375959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:54.376345 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:54.376664 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:54.377028 (+ 364us) server_negotiation.cc:299] Negotiation successful
0808 23:29:54.377274 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 23:29:54.377846 30434 ts_tablet_manager.cc:1918] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:29:54.377830 30436 ts_tablet_manager.cc:1939] T 9b7c7a47f8624673b9b9626679848b0c P b665a9d8e0534eb2a5a48ba61d21b5d8: Deleting consensus metadata
I20260808 23:29:54.378015 30434 log.cc:1196] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/9b7c7a47f8624673b9b9626679848b0c
I20260808 23:29:54.378885 30434 ts_tablet_manager.cc:1939] T 9b7c7a47f8624673b9b9626679848b0c P 9c1e3fb3cdfb442584aa6beb25ce9a83: Deleting consensus metadata
I20260808 23:29:54.379192 29915 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:29:54.379822 29608 catalog_manager.cc:4841] TS b665a9d8e0534eb2a5a48ba61d21b5d8 (127.9.44.65:42727): tablet 9b7c7a47f8624673b9b9626679848b0c (table test-table [id=0e3475bb9dc043ec96fbc61bda23cf95]) successfully deleted
I20260808 23:29:54.383939 29608 catalog_manager.cc:4841] TS 9c1e3fb3cdfb442584aa6beb25ce9a83 (127.9.44.67:38697): tablet 9b7c7a47f8624673b9b9626679848b0c (table test-table [id=0e3475bb9dc043ec96fbc61bda23cf95]) successfully deleted
W20260808 23:29:54.384383 29608 catalog_manager.cc:4808] TS 9c1e3fb3cdfb442584aa6beb25ce9a83 (127.9.44.67:38697): delete failed for tablet 9b7c7a47f8624673b9b9626679848b0c because the tablet was not found. No further retry: Not found: Tablet not found: 9b7c7a47f8624673b9b9626679848b0c
I20260808 23:29:54.466709 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 29681
I20260808 23:29:54.491390 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 29928
I20260808 23:29:54.504889 30385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:54.492242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52650 (local address 127.9.44.66:36529)
0808 23:29:54.492418 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:54.492424 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:54.492442 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:29:54.492593 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:54.492599 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:54.492671 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:54.492774 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:54.494102 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:54.494834 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:54.496305 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:54.496697 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:54.496815 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:54.499271 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:54.499283 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:54.499298 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:54.499331 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:54.502187 (+ 2856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:54.502855 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:54.502863 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:54.502865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:54.503004 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:54.503391 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:54.503395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:54.503397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:54.503799 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:54.504093 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:54.504358 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 23:29:54.504552 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 23:29:54.505510 30151 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:29:54.604773 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 29928
I20260808 23:29:54.628928 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 30165
I20260808 23:29:54.644527 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:54.629678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36206 (local address 127.9.44.67:38697)
0808 23:29:54.629989 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:54.630001 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:54.630025 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:29:54.630097 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:54.630102 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:54.630209 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:54.630342 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:54.631592 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:54.632908 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:54.634198 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:54.634882 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:54.635015 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:54.637492 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:54.637508 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:54.637534 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:54.637576 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:54.641611 (+ 4035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:54.642276 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:54.642286 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:54.642289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:54.642451 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:54.642800 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:54.642805 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:54.642808 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:54.643310 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:54.643618 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:54.643867 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 23:29:54.644095 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
I20260808 23:29:54.645308 30351 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:29:54.764961 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 30165
I20260808 23:29:54.788077 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 29584
I20260808 23:29:54.789152 29673 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:29:54.900872 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 29584
I20260808 23:29:54.931910 9393 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 23:29:55.275640 9393 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 23:29:55.275894 9393 mini_ranger.cc:66] Stopping Ranger...
I20260808 23:29:55.619511 9393 mini_ranger.cc:68] Stopped Ranger
2026-08-08 23:29:55.619 UTC [28188] LOG: received smart shutdown request
2026-08-08 23:29:55.622 UTC [28188] LOG: background worker "logical replication launcher" (PID 28202) exited with exit code 1
2026-08-08 23:29:55.622 UTC [28194] LOG: shutting down
2026-08-08 23:29:55.735 UTC [28188] LOG: database system is shut down
2026-08-08T23:29:55Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49488 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:29:55 dist-test-slave-6bbx krb5kdc[30460](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:29:55 dist-test-slave-6bbx krb5kdc[30460](info): set up 2 sockets
Aug 08 23:29:55 dist-test-slave-6bbx krb5kdc[30460](info): commencing operation
krb5kdc: starting...
W20260808 23:29:57.038806 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:29:57 dist-test-slave-6bbx krb5kdc[30460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:29:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:29:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:29:57.270223 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:43915
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:46071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:43915,127.9.44.125:35259,127.9.44.124:33201
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:29:57.596047 30479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:29:57.596328 30479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:29:57.596470 30479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:29:57.603498 30479 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:29:57.603590 30479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:29:57.603678 30479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:29:57.603749 30479 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:29:57.603808 30479 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:29:57.613137 30479 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:46071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:43915,127.9.44.125:35259,127.9.44.124:33201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:43915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:29:57.614315 30479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:29:57.615571 30479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:29:57.626791 30488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:29:57.626829 30493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:29:57.626866 30490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:29:57.627815 30479 server_base.cc:1034] running on GCE node
I20260808 23:29:57.628523 30479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:29:57.629755 30479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:29:57.630957 30479 hybrid_clock.cc:648] HybridClock initialized: now 1786231797630912 us; error 36 us; skew 500 ppm
I20260808 23:29:57.631266 30479 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:29:57 dist-test-slave-6bbx krb5kdc[30460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231797, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:29:57.634207 30479 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:29:57.635497 30479 webserver.cc:460] Webserver started at http://127.9.44.126:38549/ using document root <none> and password file <none>
I20260808 23:29:57.636186 30479 fs_manager.cc:359] Metadata directory not provided
I20260808 23:29:57.636302 30479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:29:57.636584 30479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:29:57.639374 30479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "beffe60aac0b4e1e8293077598d2a898"
format_stamp: "Formatted at 2026-08-08 23:29:57 on dist-test-slave-6bbx"
server_key: "0009561cc4efc314b31496b3862f719b"
server_key_iv: "4caf0d3d4fed4d173062735e3c6a9b09"
server_key_version: "encryptionkey@0"
I20260808 23:29:57.640156 30479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "beffe60aac0b4e1e8293077598d2a898"
format_stamp: "Formatted at 2026-08-08 23:29:57 on dist-test-slave-6bbx"
server_key: "0009561cc4efc314b31496b3862f719b"
server_key_iv: "4caf0d3d4fed4d173062735e3c6a9b09"
server_key_version: "encryptionkey@0"
I20260808 23:29:57.644889 30479 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 23:29:57.648118 30504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:57.649329 30479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 23:29:57.649492 30479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "beffe60aac0b4e1e8293077598d2a898"
format_stamp: "Formatted at 2026-08-08 23:29:57 on dist-test-slave-6bbx"
server_key: "0009561cc4efc314b31496b3862f719b"
server_key_iv: "4caf0d3d4fed4d173062735e3c6a9b09"
server_key_version: "encryptionkey@0"
I20260808 23:29:57.649668 30479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:29:57.674001 30479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:29:57.677802 30479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:29:57.678066 30479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:29:57.695415 30479 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:43915
I20260808 23:29:57.695446 30578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:43915 every 8 connection(s)
I20260808 23:29:57.696645 30479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:29:57.699479 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 30479
I20260808 23:29:57.699745 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:29:57.700623 9393 external_mini_cluster.cc:1428] Setting key 2a237c36eec5e93e993ebc99ac055bb1
I20260808 23:29:57.700646 30579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:29:57.708856 30579 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43915 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:29:57 dist-test-slave-6bbx krb5kdc[30460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231797, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.44.125@KRBTEST.COM; defaulting to no policy
I20260808 23:29:57.732299 30584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:57.713266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38416 (local address 127.9.44.126:43915)
0808 23:29:57.716123 (+ 2857us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:57.716130 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:57.716169 (+ 39us) server_negotiation.cc:407] Connection header received
0808 23:29:57.716334 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:57.716340 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:57.716595 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:57.716796 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:57.717870 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:57.719627 (+ 1757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:57.721008 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:57.721756 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:57.721957 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:57.724978 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:57.724988 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:57.725003 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:57.725042 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:57.728483 (+ 3441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:57.729153 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:57.729161 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:57.729165 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:57.729364 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:57.729740 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:57.729745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:57.729748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:57.730394 (+ 646us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:57.730585 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:57.731068 (+ 483us) server_negotiation.cc:299] Negotiation successful
0808 23:29:57.731406 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":126,"server-negotiator.queue_time_us":2697,"thread_start_us":2581,"threads_started":1}
I20260808 23:29:57.732332 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:57.712060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:43915 (local address 127.0.0.1:38416)
0808 23:29:57.712817 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:57.712870 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:57.715474 (+ 2604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:57.716821 (+ 1347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:57.716827 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:57.717338 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:29:57.717667 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:57.717684 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:57.719800 (+ 2116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:57.719804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:57.720869 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:57.720881 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:57.721956 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:57.721962 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:57.722101 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:57.722683 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:29:57.722708 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:29:57.724753 (+ 2045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:29:57.728637 (+ 3884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:57.728647 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:57.728655 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:57.729022 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:57.729480 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:57.729483 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:57.729486 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:57.729638 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:57.730516 (+ 878us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:29:57.730521 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:29:57.730702 (+ 181us) client_negotiation.cc:769] Sending connection context
0808 23:29:57.730925 (+ 223us) client_negotiation.cc:240] Negotiation successful
0808 23:29:57.731287 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":319,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.125@KRBTEST.COM" created.
I20260808 23:29:57.735217 30579 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:29:57.736100 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:57.735770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:35259 (local address 127.0.0.1:58342)
0808 23:29:57.735921 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:57.736007 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:35259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 23:29:57.736322 30507 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.44.125:35259: connect: Connection refused (error 111)
I20260808 23:29:57.737748 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:57.737420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:35259 (local address 127.0.0.1:58352)
0808 23:29:57.737574 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:57.737633 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:35259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 23:29:57.737910 30579 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.125:35259: Network error: Client connection negotiation failed: client connection to 127.9.44.125:35259: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.125.keytab.
Entry for principal kudu/127.9.44.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.125.keytab.
I20260808 23:29:57.760099 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.125:35259
--webserver_interface=127.9.44.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.125.keytab
--principal=kudu/127.9.44.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:46071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:43915,127.9.44.125:35259,127.9.44.124:33201
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:29:57.787288 30579 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } attempt: 1
I20260808 23:29:57.788343 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:57.787928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:35259 (local address 127.0.0.1:58368)
0808 23:29:57.788119 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:57.788203 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:35259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 23:29:57.789309 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:57.788979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:35259 (local address 127.0.0.1:58376)
0808 23:29:57.789109 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:57.789184 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:35259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 23:29:57.789496 30579 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.125:35259: Network error: Client connection negotiation failed: client connection to 127.9.44.125:35259: connect: Connection refused (error 111)
I20260808 23:29:57.857704 30579 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } attempt: 2
I20260808 23:29:57.858872 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:57.858396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:35259 (local address 127.0.0.1:58390)
0808 23:29:57.858627 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:57.858720 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:35259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 23:29:57.859911 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:57.859525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:35259 (local address 127.0.0.1:58392)
0808 23:29:57.859688 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:57.859775 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:35259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 23:29:57.860103 30579 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.125:35259: Network error: Client connection negotiation failed: client connection to 127.9.44.125:35259: connect: Connection refused (error 111)
I20260808 23:29:57.951336 30579 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } attempt: 3
I20260808 23:29:57.952467 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:57.952016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:35259 (local address 127.0.0.1:58394)
0808 23:29:57.952222 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:57.952317 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:35259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 23:29:57.953529 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:57.953188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:35259 (local address 127.0.0.1:58404)
0808 23:29:57.953316 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:57.953401 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:35259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 23:29:57.953703 30579 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.125:35259: Network error: Client connection negotiation failed: client connection to 127.9.44.125:35259: connect: Connection refused (error 111)
W20260808 23:29:58.079710 30587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:29:58.079964 30587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:29:58.080056 30587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:29:58.086862 30587 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:29:58.086952 30587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:29:58.087003 30587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:29:58.087056 30587 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:29:58.087100 30587 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:29:58.096230 30587 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:46071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:43915,127.9.44.125:35259,127.9.44.124:33201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.125.keytab
--principal=kudu/127.9.44.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.125:35259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.44.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:29:58.096904 30579 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } attempt: 4
I20260808 23:29:58.097362 30587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:29:58.098089 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.097539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:35259 (local address 127.0.0.1:58418)
0808 23:29:58.097842 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.097965 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:35259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
I20260808 23:29:58.098898 30587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 23:29:58.099078 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.098736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:35259 (local address 127.0.0.1:58434)
0808 23:29:58.098850 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.098945 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:35259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 23:29:58.099295 30579 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.125:35259: Network error: Client connection negotiation failed: client connection to 127.9.44.125:35259: connect: Connection refused (error 111)
W20260808 23:29:58.109830 30593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:29:58.109790 30600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:29:58.109879 30595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:29:58.110795 30587 server_base.cc:1034] running on GCE node
I20260808 23:29:58.111505 30587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:29:58.112762 30587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:29:58.113955 30587 hybrid_clock.cc:648] HybridClock initialized: now 1786231798113910 us; error 36 us; skew 500 ppm
I20260808 23:29:58.114269 30587 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:29:58 dist-test-slave-6bbx krb5kdc[30460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231798, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:29:58.124413 30587 init.cc:377] Logged in from keytab as kudu/127.9.44.125@KRBTEST.COM (short username kudu)
I20260808 23:29:58.125764 30587 webserver.cc:460] Webserver started at http://127.9.44.125:36645/ using document root <none> and password file <none>
I20260808 23:29:58.126394 30587 fs_manager.cc:359] Metadata directory not provided
I20260808 23:29:58.126510 30587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:29:58.126787 30587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:29:58.129639 30587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-1/data/instance:
uuid: "0755c21def6545db8569512c5c247afe"
format_stamp: "Formatted at 2026-08-08 23:29:58 on dist-test-slave-6bbx"
server_key: "3339a3c4f4599945464819bc85e76332"
server_key_iv: "fd9973252d6783d847321a389b88af19"
server_key_version: "encryptionkey@0"
I20260808 23:29:58.130442 30587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-1/wal/instance:
uuid: "0755c21def6545db8569512c5c247afe"
format_stamp: "Formatted at 2026-08-08 23:29:58 on dist-test-slave-6bbx"
server_key: "3339a3c4f4599945464819bc85e76332"
server_key_iv: "fd9973252d6783d847321a389b88af19"
server_key_version: "encryptionkey@0"
I20260808 23:29:58.135142 30587 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260808 23:29:58.138331 30612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:58.139529 30587 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 23:29:58.139696 30587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-1/wal
uuid: "0755c21def6545db8569512c5c247afe"
format_stamp: "Formatted at 2026-08-08 23:29:58 on dist-test-slave-6bbx"
server_key: "3339a3c4f4599945464819bc85e76332"
server_key_iv: "fd9973252d6783d847321a389b88af19"
server_key_version: "encryptionkey@0"
I20260808 23:29:58.139868 30587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:29:58.165040 30587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:29:58.168615 30587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:29:58.168870 30587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:29:58.185535 30587 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.125:35259
I20260808 23:29:58.185590 30696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.125:35259 every 8 connection(s)
I20260808 23:29:58.186733 30587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-1/data/info.pb
I20260808 23:29:58.189270 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 30587
I20260808 23:29:58.189431 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-1/wal/instance
I20260808 23:29:58.189889 9393 external_mini_cluster.cc:1428] Setting key 191389eede73b36f6c623396afcd4918
I20260808 23:29:58.190331 30697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:29:58.196521 30697 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43915 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:29:58 dist-test-slave-6bbx krb5kdc[30460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231798, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.125@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.44.124@KRBTEST.COM; defaulting to no policy
I20260808 23:29:58.215826 30584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.198969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38428 (local address 127.9.44.126:43915)
0808 23:29:58.199132 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:58.199137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:58.200371 (+ 1234us) server_negotiation.cc:407] Connection header received
0808 23:29:58.202595 (+ 2224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:58.202601 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:58.202682 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:58.202772 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:58.204296 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.204966 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.206443 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.206867 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.206980 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:58.210157 (+ 3177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:58.210164 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:58.210168 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:58.210209 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:58.212945 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.213634 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.213638 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.213641 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.213764 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.214156 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.214160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.214162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.214448 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:58.214599 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:58.215244 (+ 645us) server_negotiation.cc:299] Negotiation successful
0808 23:29:58.215472 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 23:29:58.216621 30699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.199444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:43915 (local address 127.0.0.1:38428)
0808 23:29:58.200201 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.200244 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:58.202336 (+ 2092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:58.203019 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:58.203027 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:58.203498 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:29:58.204125 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:58.204145 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.205121 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.205125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:58.206298 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:58.206312 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.207079 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.207087 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:58.207286 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:58.207902 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:29:58.207924 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:29:58.209932 (+ 2008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:29:58.213083 (+ 3151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:58.213092 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:58.213103 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:58.213511 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:58.213867 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:58.213872 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:58.213875 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:58.214041 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:58.214572 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:29:58.214579 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:29:58.214897 (+ 318us) client_negotiation.cc:769] Sending connection context
0808 23:29:58.215114 (+ 217us) client_negotiation.cc:240] Negotiation successful
0808 23:29:58.215557 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":360,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.124@KRBTEST.COM" created.
I20260808 23:29:58.218281 30697 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:29:58 dist-test-slave-6bbx krb5kdc[30460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231798, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.125@KRBTEST.COM for kudu/127.9.44.125@KRBTEST.COM
I20260808 23:29:58.231078 30699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.218809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:35259 (local address 127.0.0.1:58442)
0808 23:29:58.219022 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.219035 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:58.219130 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:58.220091 (+ 961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:58.220095 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:58.220474 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:29:58.220591 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:58.220599 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.221891 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.221895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:58.222721 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:58.222732 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.223475 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.223480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:58.223659 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:58.224169 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:29:58.224185 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:29:58.226016 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:29:58.228850 (+ 2834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:58.228855 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:58.228859 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:58.229171 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:58.229511 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:58.229516 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:58.229520 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:58.229655 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:58.230218 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:29:58.230223 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:29:58.230355 (+ 132us) client_negotiation.cc:769] Sending connection context
0808 23:29:58.230478 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 23:29:58.230723 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125}
I20260808 23:29:58.231213 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.219096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58442 (local address 127.9.44.125:35259)
0808 23:29:58.219716 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:58.219723 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:58.219748 (+ 25us) server_negotiation.cc:407] Connection header received
0808 23:29:58.219789 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:58.219794 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:58.219957 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:58.220085 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:58.220735 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.221744 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.222847 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.223342 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.223451 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:58.226204 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:58.226214 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:58.226226 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:58.226252 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:58.228731 (+ 2479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.229294 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.229302 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.229304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.229412 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.229752 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.229757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.229761 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.230094 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:58.230291 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:58.230644 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 23:29:58.230848 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":335,"threads_started":1}
I20260808 23:29:58.233522 30697 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:29:58.234557 30699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.234121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:33201 (local address 127.0.0.1:47392)
0808 23:29:58.234294 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.234426 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 23:29:58.234819 30619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
I20260808 23:29:58.236318 30699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.235965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:33201 (local address 127.0.0.1:47402)
0808 23:29:58.236103 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.236216 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 23:29:58.236531 30697 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:33201: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.124.keytab.
Entry for principal kudu/127.9.44.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.124.keytab.
I20260808 23:29:58.246193 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.124:33201
--webserver_interface=127.9.44.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.124.keytab
--principal=kudu/127.9.44.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:46071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:43915,127.9.44.125:35259,127.9.44.124:33201
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:29:58.285801 30697 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } attempt: 1
I20260808 23:29:58.286916 30699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.286480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:33201 (local address 127.0.0.1:47404)
0808 23:29:58.286659 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.286758 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 23:29:58.287871 30699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.287495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:33201 (local address 127.0.0.1:47418)
0808 23:29:58.287650 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.287752 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 23:29:58.288064 30697 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:33201: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
I20260808 23:29:58.356259 30697 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } attempt: 2
I20260808 23:29:58.357254 30699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.356824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:33201 (local address 127.0.0.1:47434)
0808 23:29:58.357017 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.357103 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 23:29:58.358191 30699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.357862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:33201 (local address 127.0.0.1:47440)
0808 23:29:58.358014 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.358080 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 23:29:58.358407 30697 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:33201: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
I20260808 23:29:58.398530 30579 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } attempt: 5
Aug 08 23:29:58 dist-test-slave-6bbx krb5kdc[30460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231797, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for kudu/127.9.44.125@KRBTEST.COM
I20260808 23:29:58.412474 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.399234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:35259 (local address 127.0.0.1:58456)
0808 23:29:58.399452 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.399475 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:58.399595 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:58.400031 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:58.400034 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:58.400409 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:29:58.400579 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:58.400588 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.401948 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.401951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:58.402810 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:58.402820 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.403839 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.403843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:58.403985 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:58.404646 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:29:58.404663 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:29:58.406621 (+ 1958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:29:58.410320 (+ 3699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:58.410324 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:58.410327 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:58.410685 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:58.411047 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:58.411051 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:58.411053 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:58.411174 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:58.411792 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:29:58.411796 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:29:58.411934 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 23:29:58.412041 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 23:29:58.412229 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 23:29:58.412693 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.399401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58456 (local address 127.9.44.125:35259)
0808 23:29:58.399603 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:58.399610 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:58.399695 (+ 85us) server_negotiation.cc:407] Connection header received
0808 23:29:58.399783 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:58.399788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:58.399891 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:58.399996 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:58.400745 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.401838 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.402969 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.403713 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.403850 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:58.406838 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:58.406845 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:58.406850 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:58.406893 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:58.410188 (+ 3295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.410794 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.410800 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.410804 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.410952 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.411267 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.411272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.411275 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.411620 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:58.411821 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:58.412165 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 23:29:58.412386 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 23:29:58.413717 30579 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:29:58.414794 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.414437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:33201 (local address 127.0.0.1:47448)
0808 23:29:58.414588 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.414660 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 23:29:58.415787 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.415443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:33201 (local address 127.0.0.1:47454)
0808 23:29:58.415580 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.415656 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 23:29:58.415997 30579 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:33201: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
I20260808 23:29:58.449599 30697 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } attempt: 3
I20260808 23:29:58.450526 30699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.450101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:33201 (local address 127.0.0.1:47456)
0808 23:29:58.450299 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.450377 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 23:29:58.451496 30699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.451084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:33201 (local address 127.0.0.1:47458)
0808 23:29:58.451302 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.451381 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260808 23:29:58.451705 30697 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:33201: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
I20260808 23:29:58.467177 30579 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } attempt: 1
I20260808 23:29:58.468148 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.467765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:33201 (local address 127.0.0.1:47466)
0808 23:29:58.467929 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.468010 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 23:29:58.469058 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.468724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:33201 (local address 127.0.0.1:47476)
0808 23:29:58.468876 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.468961 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 23:29:58.469260 30579 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:33201: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
I20260808 23:29:58.537459 30579 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } attempt: 2
I20260808 23:29:58.538511 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.538047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:33201 (local address 127.0.0.1:47492)
0808 23:29:58.538264 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.538352 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 23:29:58.539535 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.539220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:33201 (local address 127.0.0.1:47494)
0808 23:29:58.539364 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.539425 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 23:29:58.539749 30579 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:33201: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
W20260808 23:29:58.571640 30706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:29:58.571998 30706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:29:58.572144 30706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:29:58.579291 30706 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:29:58.579381 30706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:29:58.579457 30706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:29:58.579536 30706 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:29:58.579596 30706 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:29:58.588936 30706 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:46071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:43915,127.9.44.125:35259,127.9.44.124:33201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.124.keytab
--principal=kudu/127.9.44.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.124:33201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.44.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:29:58.590081 30706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:29:58.591331 30706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 23:29:58.594897 30697 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } attempt: 4
I20260808 23:29:58.596045 30699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.595470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:33201 (local address 127.0.0.1:47510)
0808 23:29:58.595733 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.595851 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 23:29:58.597261 30699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.596839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:33201 (local address 127.0.0.1:47524)
0808 23:29:58.597050 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.597132 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260808 23:29:58.597594 30697 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:33201: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
W20260808 23:29:58.602385 30713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:29:58.602339 30714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:29:58.602422 30717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:29:58.603325 30706 server_base.cc:1034] running on GCE node
I20260808 23:29:58.604058 30706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:29:58.605355 30706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:29:58.606540 30706 hybrid_clock.cc:648] HybridClock initialized: now 1786231798606489 us; error 39 us; skew 500 ppm
I20260808 23:29:58.606858 30706 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:29:58 dist-test-slave-6bbx krb5kdc[30460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231798, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:29:58.609766 30706 init.cc:377] Logged in from keytab as kudu/127.9.44.124@KRBTEST.COM (short username kudu)
I20260808 23:29:58.611037 30706 webserver.cc:460] Webserver started at http://127.9.44.124:45167/ using document root <none> and password file <none>
I20260808 23:29:58.611686 30706 fs_manager.cc:359] Metadata directory not provided
I20260808 23:29:58.611814 30706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:29:58.612103 30706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:29:58.614861 30706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-2/data/instance:
uuid: "c3c6aaa3091e4822beb3836d7c63473f"
format_stamp: "Formatted at 2026-08-08 23:29:58 on dist-test-slave-6bbx"
server_key: "515ddb12a4fc0e865dbde7f6bbc53913"
server_key_iv: "2d87c9d1066d924323c7b43ac3b832cb"
server_key_version: "encryptionkey@0"
I20260808 23:29:58.615609 30706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-2/wal/instance:
uuid: "c3c6aaa3091e4822beb3836d7c63473f"
format_stamp: "Formatted at 2026-08-08 23:29:58 on dist-test-slave-6bbx"
server_key: "515ddb12a4fc0e865dbde7f6bbc53913"
server_key_iv: "2d87c9d1066d924323c7b43ac3b832cb"
server_key_version: "encryptionkey@0"
I20260808 23:29:58.620354 30706 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 23:29:58.623560 30726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:29:58.625090 30706 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 23:29:58.625228 30706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-2/wal
uuid: "c3c6aaa3091e4822beb3836d7c63473f"
format_stamp: "Formatted at 2026-08-08 23:29:58 on dist-test-slave-6bbx"
server_key: "515ddb12a4fc0e865dbde7f6bbc53913"
server_key_iv: "2d87c9d1066d924323c7b43ac3b832cb"
server_key_version: "encryptionkey@0"
I20260808 23:29:58.625382 30706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:29:58.645944 30579 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } attempt: 3
I20260808 23:29:58.647042 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.646505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:33201 (local address 127.0.0.1:47532)
0808 23:29:58.646747 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.646850 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 23:29:58.648090 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.647653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:33201 (local address 127.0.0.1:47546)
0808 23:29:58.647879 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.647966 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260808 23:29:58.648329 30579 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:33201: Network error: Client connection negotiation failed: client connection to 127.9.44.124:33201: connect: Connection refused (error 111)
I20260808 23:29:58.649230 30706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:29:58.652765 30706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:29:58.652982 30706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:29:58.669801 30706 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.124:33201
I20260808 23:29:58.669838 30801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.124:33201 every 8 connection(s)
I20260808 23:29:58.671051 30706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-2/data/info.pb
I20260808 23:29:58.674592 30802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:29:58.677838 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 30706
I20260808 23:29:58.678042 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786231597224934-9393-0/minicluster-data/master-2/wal/instance
I20260808 23:29:58.678612 9393 external_mini_cluster.cc:1428] Setting key 7b77f1388ed624ac7797cddc91ef1339
I20260808 23:29:58.682976 30802 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43915 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:29:58 dist-test-slave-6bbx krb5kdc[30460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.124@KRBTEST.COM
Aug 08 23:29:58 dist-test-slave-6bbx krb5kdc[30460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231798, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.124@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:29:58.699872 30584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.685307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38442 (local address 127.9.44.126:43915)
0808 23:29:58.685533 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:58.685539 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:58.686331 (+ 792us) server_negotiation.cc:407] Connection header received
0808 23:29:58.686509 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:58.686513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:58.686575 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:58.686649 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:58.687971 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.688633 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.690017 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.690429 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.690535 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:58.693875 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:58.693880 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:58.693883 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:58.693914 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:58.696631 (+ 2717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.697363 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.697368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.697372 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.697537 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.697986 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.697991 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.697994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.698294 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:58.698437 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:58.699251 (+ 814us) server_negotiation.cc:299] Negotiation successful
0808 23:29:58.699482 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 23:29:58.701164 30805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.680421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47558 (local address 127.9.44.124:33201)
0808 23:29:58.681319 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:58.681334 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:58.681383 (+ 49us) server_negotiation.cc:407] Connection header received
0808 23:29:58.682990 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:58.683013 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:58.683375 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:58.683745 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:58.685310 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.687255 (+ 1945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.688658 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.689399 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.689560 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:58.693694 (+ 4134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:58.693702 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:58.693715 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:58.693744 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:58.696948 (+ 3204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.697727 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.697734 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.697737 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.697928 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.698309 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.698313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.698315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.698823 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:58.699096 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:58.699473 (+ 377us) server_negotiation.cc:299] Negotiation successful
0808 23:29:58.699898 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":671,"spinlock_wait_cycles":16256,"thread_start_us":450,"threads_started":1}
I20260808 23:29:58.701186 30807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.685379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:43915 (local address 127.0.0.1:38442)
0808 23:29:58.686187 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.686242 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:58.686383 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:58.686745 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:58.686753 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:58.687509 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:29:58.687827 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:58.687838 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.688788 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.688792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:58.689880 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:58.689893 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.690651 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.690659 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:58.690795 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:58.691331 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:29:58.691357 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:29:58.693698 (+ 2341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:29:58.696753 (+ 3055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:58.696764 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:58.696779 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:58.697237 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:58.697658 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:58.697663 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:58.697667 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:58.697889 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:58.698417 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:29:58.698424 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:29:58.698823 (+ 399us) client_negotiation.cc:769] Sending connection context
0808 23:29:58.700125 (+ 1302us) client_negotiation.cc:240] Negotiation successful
0808 23:29:58.700456 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"mutex_wait_us":2,"thread_start_us":384,"threads_started":1}
I20260808 23:29:58.702665 30802 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:29:58 dist-test-slave-6bbx krb5kdc[30460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231798, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.124@KRBTEST.COM for kudu/127.9.44.125@KRBTEST.COM
Aug 08 23:29:58 dist-test-slave-6bbx krb5kdc[30460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:29:58.716325 30807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.703198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:35259 (local address 127.0.0.1:58462)
0808 23:29:58.703327 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.703343 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:58.703417 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:58.703885 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:58.703888 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:58.704183 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:29:58.704329 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:58.704340 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.705650 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.705655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:58.706998 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:58.707011 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.708020 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.708026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:58.708292 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:58.708793 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:29:58.708814 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:29:58.710973 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:29:58.714121 (+ 3148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:58.714126 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:58.714129 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:58.714507 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:58.714928 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:58.714931 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:58.714934 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:58.715072 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:58.715600 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:29:58.715603 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:29:58.715775 (+ 172us) client_negotiation.cc:769] Sending connection context
0808 23:29:58.715913 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 23:29:58.716092 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 23:29:58.716604 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.703392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58462 (local address 127.9.44.125:35259)
0808 23:29:58.703567 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:58.703573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:58.703593 (+ 20us) server_negotiation.cc:407] Connection header received
0808 23:29:58.703654 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:58.703658 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:58.703760 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:58.703862 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:58.704473 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.705500 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.707204 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.707868 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.707966 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:58.711167 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:58.711174 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:58.711179 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:58.711216 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:58.713993 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.714637 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.714643 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.714646 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.714810 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.715174 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.715179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.715183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.715497 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:58.715651 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:58.716037 (+ 386us) server_negotiation.cc:299] Negotiation successful
0808 23:29:58.716249 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 23:29:58.717501 30802 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:29:58.719939 30584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.704825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38446 (local address 127.9.44.126:43915)
0808 23:29:58.704966 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:58.704971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:58.704988 (+ 17us) server_negotiation.cc:407] Connection header received
0808 23:29:58.705032 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:58.705036 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:58.705096 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:58.705180 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:58.706753 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.707801 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.709495 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.709906 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.710023 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:58.714287 (+ 4264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:58.714291 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:58.714294 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:58.714322 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:58.717329 (+ 3007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.718052 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.718057 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.718060 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.718223 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.718630 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.718633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.718635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.718898 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:58.719160 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:58.719445 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 23:29:58.719644 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 23:29:58 dist-test-slave-6bbx krb5kdc[30460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231798, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.124@KRBTEST.COM for kudu/127.9.44.124@KRBTEST.COM
Aug 08 23:29:58 dist-test-slave-6bbx krb5kdc[30460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.125@KRBTEST.COM
I20260808 23:29:58.730752 30807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.718198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:33201 (local address 127.0.0.1:47560)
0808 23:29:58.718356 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.718373 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:58.718451 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:58.718736 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:58.718741 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:58.719083 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:29:58.719201 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:58.719209 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.720278 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.720284 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:58.721411 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:58.721427 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.722151 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.722158 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:58.722332 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:58.722846 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:29:58.722864 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:29:58.724758 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:29:58.728067 (+ 3309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:58.728071 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:58.728073 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:58.728421 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:58.728829 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:58.728832 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:58.728834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:58.728979 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:58.730048 (+ 1069us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:29:58.730052 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:29:58.730171 (+ 119us) client_negotiation.cc:769] Sending connection context
0808 23:29:58.730302 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 23:29:58.730502 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 23:29:58.731038 30805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.718373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47560 (local address 127.9.44.124:33201)
0808 23:29:58.718513 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:58.718518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:58.718541 (+ 23us) server_negotiation.cc:407] Connection header received
0808 23:29:58.718577 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:58.718581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:58.718631 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:58.718705 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:58.719323 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.720171 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.721559 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.722007 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.722113 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:58.724953 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:58.724960 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:58.724965 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:58.725000 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:58.727941 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.728578 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.728584 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.728587 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.728729 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.729096 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.729101 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.729104 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.729446 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:58.729598 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:58.730434 (+ 836us) server_negotiation.cc:299] Negotiation successful
0808 23:29:58.730667 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 23:29:58.736393 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.721555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58478 (local address 127.9.44.125:35259)
0808 23:29:58.721710 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:58.721715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:58.721732 (+ 17us) server_negotiation.cc:407] Connection header received
0808 23:29:58.721885 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:58.721890 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:58.721950 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:58.722024 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:58.723311 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.724199 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.725759 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.726160 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.726258 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:58.730989 (+ 4731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:58.730995 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:58.730999 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:58.731037 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:58.733942 (+ 2905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.734583 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.734589 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.734593 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.734741 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.735063 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.735068 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.735071 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.735403 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:58.735689 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:58.735867 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 23:29:58.736080 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 23:29:58.736801 30802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3c6aaa3091e4822beb3836d7c63473f: Bootstrap starting.
I20260808 23:29:58.740255 30802 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3c6aaa3091e4822beb3836d7c63473f: Neither blocks nor log segments found. Creating new log.
I20260808 23:29:58.740849 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:29:58.741272 30802 log.cc:824] T 00000000000000000000000000000000 P c3c6aaa3091e4822beb3836d7c63473f: Log is configured to *not* fsync() on all Append() calls
I20260808 23:29:58.744019 30802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3c6aaa3091e4822beb3836d7c63473f: No bootstrap required, opened a new log
I20260808 23:29:58.752153 30802 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3c6aaa3091e4822beb3836d7c63473f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beffe60aac0b4e1e8293077598d2a898" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43915 } } peers { permanent_uuid: "0755c21def6545db8569512c5c247afe" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } } peers { permanent_uuid: "c3c6aaa3091e4822beb3836d7c63473f" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } }
I20260808 23:29:58.752761 30802 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3c6aaa3091e4822beb3836d7c63473f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:29:58.752987 30802 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3c6aaa3091e4822beb3836d7c63473f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3c6aaa3091e4822beb3836d7c63473f, State: Initialized, Role: FOLLOWER
I20260808 23:29:58.753916 30802 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3c6aaa3091e4822beb3836d7c63473f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beffe60aac0b4e1e8293077598d2a898" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43915 } } peers { permanent_uuid: "0755c21def6545db8569512c5c247afe" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } } peers { permanent_uuid: "c3c6aaa3091e4822beb3836d7c63473f" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } }
I20260808 23:29:58.761075 30820 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3c6aaa3091e4822beb3836d7c63473f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beffe60aac0b4e1e8293077598d2a898" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43915 } } peers { permanent_uuid: "0755c21def6545db8569512c5c247afe" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } } peers { permanent_uuid: "c3c6aaa3091e4822beb3836d7c63473f" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } } }
I20260808 23:29:58.763375 30820 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3c6aaa3091e4822beb3836d7c63473f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:29:58.763248 30802 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3c6aaa3091e4822beb3836d7c63473f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:29:58.768203 30584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.749878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38456 (local address 127.9.44.126:43915)
0808 23:29:58.750070 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:58.750076 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:58.750094 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:29:58.750144 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:58.750149 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:58.750210 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:58.750288 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:58.753123 (+ 2835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.754332 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.756564 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.757260 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.757357 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:58.761216 (+ 3859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:58.761223 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:58.761228 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:58.761267 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:58.764781 (+ 3514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.765709 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.765715 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.765718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.765895 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.766347 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.766353 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.766355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.766709 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:58.766949 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:58.767582 (+ 633us) server_negotiation.cc:299] Negotiation successful
0808 23:29:58.767837 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 23:29:58.774951 30805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.749697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47568 (local address 127.9.44.124:33201)
0808 23:29:58.750526 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:58.750531 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:58.758616 (+ 8085us) server_negotiation.cc:407] Connection header received
0808 23:29:58.758866 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:58.758871 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:58.758945 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:58.759010 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:58.760891 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.761962 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.764066 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.764791 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.764874 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:58.768115 (+ 3241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:58.768121 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:58.768125 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:58.768160 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:58.771545 (+ 3385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.772576 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.772581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.772585 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.772761 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.773213 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.773218 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.773222 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.773587 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:58.773795 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:58.774405 (+ 610us) server_negotiation.cc:299] Negotiation successful
0808 23:29:58.774605 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":712}
W20260808 23:29:58.785928 30835 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c3c6aaa3091e4822beb3836d7c63473f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:29:58.786106 30835 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:29:58.786201 30701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.751006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58484 (local address 127.9.44.125:35259)
0808 23:29:58.751163 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:58.751169 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:58.770768 (+ 19599us) server_negotiation.cc:407] Connection header received
0808 23:29:58.771004 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:58.771009 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:58.771085 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:58.771151 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:58.772728 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.773741 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.775521 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.776128 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.776223 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:58.779340 (+ 3117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:58.779345 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:58.779348 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:58.779378 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:58.782297 (+ 2919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.783257 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.783262 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.783265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.783422 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.783943 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.783948 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.783951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.784284 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:58.785499 (+ 1215us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:58.785634 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 23:29:58.785864 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 23:29:58.783254 30706 master_runner.cc:186] Error getting master registration for 127.9.44.126:43915: OK, instance_id { permanent_uuid: "beffe60aac0b4e1e8293077598d2a898" instance_seqno: 1786231797690582 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 23:29:58.793144 30706 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 23:29:58.825537 30579 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } attempt: 4
Aug 08 23:29:58 dist-test-slave-6bbx krb5kdc[30460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231797, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for kudu/127.9.44.124@KRBTEST.COM
I20260808 23:29:58.840518 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.826056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:33201 (local address 127.0.0.1:47584)
0808 23:29:58.826277 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.826306 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:58.826456 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:58.826800 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:58.826804 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:58.827325 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:29:58.827546 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:58.827560 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.828983 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.828989 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:58.830513 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:58.830531 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.831237 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.831244 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:58.831442 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:58.832046 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:29:58.832070 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:29:58.834680 (+ 2610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:29:58.838190 (+ 3510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:58.838196 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:58.838199 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:58.838551 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:58.839000 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:58.839004 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:58.839007 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:58.839144 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:58.839763 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:29:58.839767 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:29:58.839908 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 23:29:58.840018 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 23:29:58.840248 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 23:29:58.840734 30805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.826153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47584 (local address 127.9.44.124:33201)
0808 23:29:58.826345 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:58.826351 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:58.826369 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:29:58.826581 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:58.826587 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:58.826668 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:58.826755 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:58.827745 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.828849 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.830651 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.831089 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.831177 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:58.834854 (+ 3677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:58.834859 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:58.834862 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:58.834891 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:58.838040 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.838668 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.838673 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.838677 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.838856 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.839252 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.839257 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.839261 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.839618 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:58.839816 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:58.840136 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 23:29:58.840377 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 23:29:58.846627 30579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898: Bootstrap starting.
I20260808 23:29:58.850723 30579 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898: Neither blocks nor log segments found. Creating new log.
I20260808 23:29:58.852080 30579 log.cc:824] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898: Log is configured to *not* fsync() on all Append() calls
I20260808 23:29:58.855155 30579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898: No bootstrap required, opened a new log
I20260808 23:29:58.866240 30579 raft_consensus.cc:348] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beffe60aac0b4e1e8293077598d2a898" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43915 } } peers { permanent_uuid: "0755c21def6545db8569512c5c247afe" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } } peers { permanent_uuid: "c3c6aaa3091e4822beb3836d7c63473f" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } }
I20260808 23:29:58.866611 30579 raft_consensus.cc:374] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:29:58.866744 30579 raft_consensus.cc:729] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beffe60aac0b4e1e8293077598d2a898, State: Initialized, Role: FOLLOWER
I20260808 23:29:58.867355 30579 consensus_queue.cc:260] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beffe60aac0b4e1e8293077598d2a898" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43915 } } peers { permanent_uuid: "0755c21def6545db8569512c5c247afe" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } } peers { permanent_uuid: "c3c6aaa3091e4822beb3836d7c63473f" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } }
I20260808 23:29:58.868307 30838 sys_catalog.cc:455] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beffe60aac0b4e1e8293077598d2a898" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43915 } } peers { permanent_uuid: "0755c21def6545db8569512c5c247afe" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } } peers { permanent_uuid: "c3c6aaa3091e4822beb3836d7c63473f" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } } }
I20260808 23:29:58.868631 30838 sys_catalog.cc:458] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:29:58.869596 30579 sys_catalog.cc:564] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 23:29:58.879089 30853 catalog_manager.cc:1558] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:29:58.879294 30853 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:29:58.878960 30479 master_runner.cc:186] Error getting master registration for 127.9.44.125:35259: OK, instance_id { permanent_uuid: "0755c21def6545db8569512c5c247afe" instance_seqno: 1786231798180776 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 23:29:58.886054 30479 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 23:29:58.896797 30697 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } attempt: 5
W20260808 23:29:58.905014 30732 tablet.cc:2369] T 00000000000000000000000000000000 P c3c6aaa3091e4822beb3836d7c63473f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 23:29:58 dist-test-slave-6bbx krb5kdc[30460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231798, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.125@KRBTEST.COM for kudu/127.9.44.124@KRBTEST.COM
I20260808 23:29:58.911497 30699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.897269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:33201 (local address 127.0.0.1:47590)
0808 23:29:58.897516 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 23:29:58.897535 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:29:58.897647 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:29:58.897939 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:29:58.897944 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:29:58.898510 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:29:58.898697 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:58.898707 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.900037 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.900043 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:58.900932 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:29:58.900945 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.901678 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.901684 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:29:58.901894 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:58.903711 (+ 1817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:29:58.903743 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:29:58.906296 (+ 2553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:29:58.909224 (+ 2928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:58.909230 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:58.909234 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:58.909600 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:58.909951 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:29:58.909957 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:29:58.909959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:29:58.910086 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:29:58.910576 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:29:58.910581 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:29:58.910729 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 23:29:58.910851 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 23:29:58.911122 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134}
I20260808 23:29:58.911530 30805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:29:58.897354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47590 (local address 127.9.44.124:33201)
0808 23:29:58.897519 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 23:29:58.897525 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:29:58.897599 (+ 74us) server_negotiation.cc:407] Connection header received
0808 23:29:58.897757 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:29:58.897763 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:29:58.897822 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:29:58.897900 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:29:58.898859 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.899882 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.901076 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:29:58.901542 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:29:58.901650 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:29:58.906476 (+ 4826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:29:58.906481 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:29:58.906484 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:29:58.906513 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:29:58.909097 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.909715 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.909721 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.909723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.909840 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:29:58.910190 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:29:58.910195 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:29:58.910197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:29:58.910463 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:29:58.910599 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 23:29:58.910962 (+ 363us) server_negotiation.cc:299] Negotiation successful
0808 23:29:58.911178 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 23:29:58.916107 30697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0755c21def6545db8569512c5c247afe: Bootstrap starting.
I20260808 23:29:58.919131 30697 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0755c21def6545db8569512c5c247afe: Neither blocks nor log segments found. Creating new log.
I20260808 23:29:58.920176 30697 log.cc:824] T 00000000000000000000000000000000 P 0755c21def6545db8569512c5c247afe: Log is configured to *not* fsync() on all Append() calls
I20260808 23:29:58.924710 30697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0755c21def6545db8569512c5c247afe: No bootstrap required, opened a new log
W20260808 23:29:58.930682 30515 tablet.cc:2369] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:29:58.932156 30697 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0755c21def6545db8569512c5c247afe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beffe60aac0b4e1e8293077598d2a898" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43915 } } peers { permanent_uuid: "0755c21def6545db8569512c5c247afe" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } } peers { permanent_uuid: "c3c6aaa3091e4822beb3836d7c63473f" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } }
I20260808 23:29:58.932452 30697 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0755c21def6545db8569512c5c247afe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:29:58.932565 30697 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0755c21def6545db8569512c5c247afe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0755c21def6545db8569512c5c247afe, State: Initialized, Role: FOLLOWER
I20260808 23:29:58.933097 30697 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0755c21def6545db8569512c5c247afe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beffe60aac0b4e1e8293077598d2a898" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43915 } } peers { permanent_uuid: "0755c21def6545db8569512c5c247afe" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } } peers { permanent_uuid: "c3c6aaa3091e4822beb3836d7c63473f" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } }
I20260808 23:29:58.933944 30859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0755c21def6545db8569512c5c247afe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beffe60aac0b4e1e8293077598d2a898" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43915 } } peers { permanent_uuid: "0755c21def6545db8569512c5c247afe" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } } peers { permanent_uuid: "c3c6aaa3091e4822beb3836d7c63473f" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } } }
I20260808 23:29:58.934368 30859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0755c21def6545db8569512c5c247afe [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:29:58.935039 30697 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0755c21def6545db8569512c5c247afe [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:29:58.950249 30587 master_runner.cc:263] No leader master found from master 0755c21def6545db8569512c5c247afe
I20260808 23:29:58.950394 30587 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 23:29:58.952960 30873 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0755c21def6545db8569512c5c247afe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:29:58.953151 30873 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:29:59.024287 30838 raft_consensus.cc:482] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:29:59.024677 30838 raft_consensus.cc:504] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beffe60aac0b4e1e8293077598d2a898" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43915 } } peers { permanent_uuid: "0755c21def6545db8569512c5c247afe" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } } peers { permanent_uuid: "c3c6aaa3091e4822beb3836d7c63473f" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } }
I20260808 23:29:59.026355 30838 leader_election.cc:288] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0755c21def6545db8569512c5c247afe (127.9.44.125:35259), c3c6aaa3091e4822beb3836d7c63473f (127.9.44.124:33201)
I20260808 23:29:59.027609 30764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "beffe60aac0b4e1e8293077598d2a898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3c6aaa3091e4822beb3836d7c63473f" is_pre_election: true
I20260808 23:29:59.028256 30764 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c3c6aaa3091e4822beb3836d7c63473f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate beffe60aac0b4e1e8293077598d2a898 in term 0.
I20260808 23:29:59.029048 30512 leader_election.cc:302] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: beffe60aac0b4e1e8293077598d2a898, c3c6aaa3091e4822beb3836d7c63473f; no voters:
I20260808 23:29:59.030292 30662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "beffe60aac0b4e1e8293077598d2a898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0755c21def6545db8569512c5c247afe" is_pre_election: true
I20260808 23:29:59.030835 30662 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0755c21def6545db8569512c5c247afe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate beffe60aac0b4e1e8293077598d2a898 in term 0.
I20260808 23:29:59.031778 30838 raft_consensus.cc:2781] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:29:59.031931 30838 raft_consensus.cc:482] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:29:59.032042 30838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:29:59.035380 30838 raft_consensus.cc:504] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beffe60aac0b4e1e8293077598d2a898" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43915 } } peers { permanent_uuid: "0755c21def6545db8569512c5c247afe" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } } peers { permanent_uuid: "c3c6aaa3091e4822beb3836d7c63473f" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } }
I20260808 23:29:59.039960 30662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "beffe60aac0b4e1e8293077598d2a898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0755c21def6545db8569512c5c247afe"
I20260808 23:29:59.040151 30662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0755c21def6545db8569512c5c247afe [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:29:59.043947 30662 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0755c21def6545db8569512c5c247afe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate beffe60aac0b4e1e8293077598d2a898 in term 1.
I20260808 23:29:59.044458 30507 leader_election.cc:302] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0755c21def6545db8569512c5c247afe, beffe60aac0b4e1e8293077598d2a898; no voters:
I20260808 23:29:59.046514 30764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "beffe60aac0b4e1e8293077598d2a898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3c6aaa3091e4822beb3836d7c63473f"
I20260808 23:29:59.046713 30764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3c6aaa3091e4822beb3836d7c63473f [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:29:59.050122 30764 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c3c6aaa3091e4822beb3836d7c63473f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate beffe60aac0b4e1e8293077598d2a898 in term 1.
I20260808 23:29:59.050832 30838 leader_election.cc:288] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [CANDIDATE]: Term 1 election: Requested vote from peers 0755c21def6545db8569512c5c247afe (127.9.44.125:35259), c3c6aaa3091e4822beb3836d7c63473f (127.9.44.124:33201)
I20260808 23:29:59.051235 30838 raft_consensus.cc:2781] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:29:59.051626 30838 raft_consensus.cc:686] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [term 1 LEADER]: Becoming Leader. State: Replica: beffe60aac0b4e1e8293077598d2a898, State: Running, Role: LEADER
I20260808 23:29:59.052938 30838 consensus_queue.cc:237] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beffe60aac0b4e1e8293077598d2a898" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43915 } } peers { permanent_uuid: "0755c21def6545db8569512c5c247afe" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } } peers { permanent_uuid: "c3c6aaa3091e4822beb3836d7c63473f" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } }
I20260808 23:29:59.071514 30875 sys_catalog.cc:455] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [sys.catalog]: SysCatalogTable state changed. Reason: New leader beffe60aac0b4e1e8293077598d2a898. Latest consensus state: current_term: 1 leader_uuid: "beffe60aac0b4e1e8293077598d2a898" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beffe60aac0b4e1e8293077598d2a898" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43915 } } peers { permanent_uuid: "0755c21def6545db8569512c5c247afe" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } } peers { permanent_uuid: "c3c6aaa3091e4822beb3836d7c63473f" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } } }
I20260808 23:29:59.072118 30875 sys_catalog.cc:458] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [sys.catalog]: This master's current role is: LEADER
I20260808 23:29:59.086843 30879 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:29:59.089331 30879 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:29:59.111806 30764 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c3c6aaa3091e4822beb3836d7c63473f [term 1 FOLLOWER]: Refusing update from remote peer beffe60aac0b4e1e8293077598d2a898: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:29:59.112957 30875 consensus_queue.cc:1035] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3c6aaa3091e4822beb3836d7c63473f" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:29:59.120044 30662 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0755c21def6545db8569512c5c247afe [term 1 FOLLOWER]: Refusing update from remote peer beffe60aac0b4e1e8293077598d2a898: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:29:59.120913 30875 consensus_queue.cc:1035] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0755c21def6545db8569512c5c247afe" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:29:59.136850 30820 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3c6aaa3091e4822beb3836d7c63473f [sys.catalog]: SysCatalogTable state changed. Reason: New leader beffe60aac0b4e1e8293077598d2a898. Latest consensus state: current_term: 1 leader_uuid: "beffe60aac0b4e1e8293077598d2a898" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beffe60aac0b4e1e8293077598d2a898" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43915 } } peers { permanent_uuid: "0755c21def6545db8569512c5c247afe" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } } peers { permanent_uuid: "c3c6aaa3091e4822beb3836d7c63473f" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } } }
I20260808 23:29:59.137177 30820 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3c6aaa3091e4822beb3836d7c63473f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:29:59.146652 30880 mvcc.cc:205] Tried to move back new op lower bound from 7316405449148866560 to 7316405448931991552. Current Snapshot: MvccSnapshot[applied={T|T < 7316405449148866560}]
I20260808 23:29:59.146917 30882 mvcc.cc:205] Tried to move back new op lower bound from 7316405449148866560 to 7316405448931991552. Current Snapshot: MvccSnapshot[applied={T|T < 7316405449148866560}]
I20260808 23:29:59.152535 30859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0755c21def6545db8569512c5c247afe [sys.catalog]: SysCatalogTable state changed. Reason: New leader beffe60aac0b4e1e8293077598d2a898. Latest consensus state: current_term: 1 leader_uuid: "beffe60aac0b4e1e8293077598d2a898" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beffe60aac0b4e1e8293077598d2a898" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43915 } } peers { permanent_uuid: "0755c21def6545db8569512c5c247afe" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } } peers { permanent_uuid: "c3c6aaa3091e4822beb3836d7c63473f" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } } }
I20260808 23:29:59.152853 30859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0755c21def6545db8569512c5c247afe [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:29:59.157979 30883 mvcc.cc:205] Tried to move back new op lower bound from 7316405449148866560 to 7316405448931991552. Current Snapshot: MvccSnapshot[applied={T|T < 7316405449148866560}]
I20260808 23:29:59.158908 30820 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3c6aaa3091e4822beb3836d7c63473f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "beffe60aac0b4e1e8293077598d2a898" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beffe60aac0b4e1e8293077598d2a898" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43915 } } peers { permanent_uuid: "0755c21def6545db8569512c5c247afe" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } } peers { permanent_uuid: "c3c6aaa3091e4822beb3836d7c63473f" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } } }
I20260808 23:29:59.159137 30820 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3c6aaa3091e4822beb3836d7c63473f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:29:59.161249 30838 sys_catalog.cc:455] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "beffe60aac0b4e1e8293077598d2a898" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beffe60aac0b4e1e8293077598d2a898" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43915 } } peers { permanent_uuid: "0755c21def6545db8569512c5c247afe" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } } peers { permanent_uuid: "c3c6aaa3091e4822beb3836d7c63473f" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } } }
I20260808 23:29:59.161571 30838 sys_catalog.cc:458] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [sys.catalog]: This master's current role is: LEADER
I20260808 23:29:59.161906 30838 sys_catalog.cc:455] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "beffe60aac0b4e1e8293077598d2a898" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beffe60aac0b4e1e8293077598d2a898" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43915 } } peers { permanent_uuid: "0755c21def6545db8569512c5c247afe" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } } peers { permanent_uuid: "c3c6aaa3091e4822beb3836d7c63473f" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } } }
I20260808 23:29:59.162201 30838 sys_catalog.cc:458] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898 [sys.catalog]: This master's current role is: LEADER
I20260808 23:29:59.162879 30859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0755c21def6545db8569512c5c247afe [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "beffe60aac0b4e1e8293077598d2a898" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "beffe60aac0b4e1e8293077598d2a898" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 43915 } } peers { permanent_uuid: "0755c21def6545db8569512c5c247afe" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 35259 } } peers { permanent_uuid: "c3c6aaa3091e4822beb3836d7c63473f" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 33201 } } }
I20260808 23:29:59.163110 30859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0755c21def6545db8569512c5c247afe [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:29:59.164521 30879 catalog_manager.cc:1347] Generated new cluster ID: fbc9aa79f06d42ba9de69ff83bba7dbe
I20260808 23:29:59.164618 30879 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:29:59.197533 30879 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:29:59.199431 30879 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:29:59.216463 30879 catalog_manager.cc:5868] T 00000000000000000000000000000000 P beffe60aac0b4e1e8293077598d2a898: Generated new TSK 0
I20260808 23:29:59.217484 30879 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:29:59.788105 30835 catalog_manager.cc:1259] Loaded cluster ID: fbc9aa79f06d42ba9de69ff83bba7dbe
I20260808 23:29:59.788208 30835 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c3c6aaa3091e4822beb3836d7c63473f: loading cluster ID for follower catalog manager: success
I20260808 23:29:59.790625 30835 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c3c6aaa3091e4822beb3836d7c63473f: acquiring CA information for follower catalog manager: success
I20260808 23:29:59.792405 30835 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c3c6aaa3091e4822beb3836d7c63473f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 23:29:59.955161 30873 catalog_manager.cc:1259] Loaded cluster ID: fbc9aa79f06d42ba9de69ff83bba7dbe
I20260808 23:29:59.955274 30873 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0755c21def6545db8569512c5c247afe: loading cluster ID for follower catalog manager: success
I20260808 23:29:59.957736 30873 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0755c21def6545db8569512c5c247afe: acquiring CA information for follower catalog manager: success
I20260808 23:29:59.959479 30873 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0755c21def6545db8569512c5c247afe: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 23:30:00.365927 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 30479
I20260808 23:30:00.367215 30573 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:30:00.474264 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 30479
I20260808 23:30:00.500905 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 30587
I20260808 23:30:00.502043 30691 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:30:00.607811 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 30587
I20260808 23:30:00.633414 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 30706
I20260808 23:30:00.634580 30792 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:30:00.740466 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 30706
2026-08-08T23:30:00Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4899 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:30:00 dist-test-slave-6bbx krb5kdc[30904](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:30:00 dist-test-slave-6bbx krb5kdc[30904](info): set up 2 sockets
Aug 08 23:30:00 dist-test-slave-6bbx krb5kdc[30904](info): commencing operation
krb5kdc: starting...
W20260808 23:30:01.942229 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:30:02 dist-test-slave-6bbx krb5kdc[30904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:30:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:30:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:30:02.172853 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:45491
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:45961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:45491
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:30:02.500263 30925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:30:02.500514 30925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:30:02.500603 30925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:30:02.507387 30925 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:30:02.507478 30925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:30:02.507529 30925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:30:02.507584 30925 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:30:02.507627 30925 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:30:02.516861 30925 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:45961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:45491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:45491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:30:02.517901 30925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:30:02.519039 30925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:30:02.529855 30934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:02.529919 30933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:02.529919 30936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:30:02.530655 30925 server_base.cc:1034] running on GCE node
I20260808 23:30:02.531353 30925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:30:02.532661 30925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:30:02.533855 30925 hybrid_clock.cc:648] HybridClock initialized: now 1786231802533800 us; error 42 us; skew 500 ppm
I20260808 23:30:02.534168 30925 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:30:02 dist-test-slave-6bbx krb5kdc[30904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231802, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:30:02.537147 30925 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:30:02.538466 30925 webserver.cc:460] Webserver started at http://127.9.44.126:46769/ using document root <none> and password file <none>
I20260808 23:30:02.539090 30925 fs_manager.cc:359] Metadata directory not provided
I20260808 23:30:02.539206 30925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:30:02.539516 30925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:30:02.542351 30925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "f6a649fe6c98494e90c82b8d44568eea"
format_stamp: "Formatted at 2026-08-08 23:30:02 on dist-test-slave-6bbx"
server_key: "e422befe2431eacce175833bddab64c7"
server_key_iv: "80ba9c11f124ca8b4d61401f52b5d735"
server_key_version: "encryptionkey@0"
I20260808 23:30:02.543073 30925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "f6a649fe6c98494e90c82b8d44568eea"
format_stamp: "Formatted at 2026-08-08 23:30:02 on dist-test-slave-6bbx"
server_key: "e422befe2431eacce175833bddab64c7"
server_key_iv: "80ba9c11f124ca8b4d61401f52b5d735"
server_key_version: "encryptionkey@0"
I20260808 23:30:02.547894 30925 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260808 23:30:02.551107 30945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:02.552325 30925 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:30:02.552484 30925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "f6a649fe6c98494e90c82b8d44568eea"
format_stamp: "Formatted at 2026-08-08 23:30:02 on dist-test-slave-6bbx"
server_key: "e422befe2431eacce175833bddab64c7"
server_key_iv: "80ba9c11f124ca8b4d61401f52b5d735"
server_key_version: "encryptionkey@0"
I20260808 23:30:02.552654 30925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:30:02.571034 30925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:30:02.574560 30925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:30:02.574822 30925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:30:02.592350 30925 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:45491
I20260808 23:30:02.592384 31027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:45491 every 8 connection(s)
I20260808 23:30:02.593611 30925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:30:02.597039 31028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:30:02.598115 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 30925
I20260808 23:30:02.598344 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:30:02.599072 9393 external_mini_cluster.cc:1428] Setting key ce0894d40e1bc0e6cb5fa911f7814eed
I20260808 23:30:02.608933 31028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6a649fe6c98494e90c82b8d44568eea: Bootstrap starting.
I20260808 23:30:02.611961 31028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6a649fe6c98494e90c82b8d44568eea: Neither blocks nor log segments found. Creating new log.
Aug 08 23:30:02 dist-test-slave-6bbx krb5kdc[30904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:30:02.612918 31028 log.cc:824] T 00000000000000000000000000000000 P f6a649fe6c98494e90c82b8d44568eea: Log is configured to *not* fsync() on all Append() calls
I20260808 23:30:02.615788 31028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6a649fe6c98494e90c82b8d44568eea: No bootstrap required, opened a new log
I20260808 23:30:02.619930 31033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:02.600828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40884 (local address 127.9.44.126:45491)
0808 23:30:02.601558 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:02.601570 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:02.601611 (+ 41us) server_negotiation.cc:407] Connection header received
0808 23:30:02.603211 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:02.603233 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:02.603598 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:02.604009 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:02.605463 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:02.606760 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:02.608108 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:02.608753 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:02.608934 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:02.612992 (+ 4058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:02.613002 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:02.613021 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:02.613070 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:02.616057 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:02.616710 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:02.616716 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:02.616722 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:02.616892 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:02.617268 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:02.617272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:02.617274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:02.617790 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:02.618113 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:02.618543 (+ 430us) server_negotiation.cc:299] Negotiation successful
0808 23:30:02.618934 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":314,"threads_started":1}
I20260808 23:30:02.623410 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:30:02.623356 31028 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6a649fe6c98494e90c82b8d44568eea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a649fe6c98494e90c82b8d44568eea" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 45491 } }
I20260808 23:30:02.623725 31028 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6a649fe6c98494e90c82b8d44568eea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:30:02.623821 31028 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6a649fe6c98494e90c82b8d44568eea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6a649fe6c98494e90c82b8d44568eea, State: Initialized, Role: FOLLOWER
I20260808 23:30:02.624317 31028 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6a649fe6c98494e90c82b8d44568eea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a649fe6c98494e90c82b8d44568eea" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 45491 } }
I20260808 23:30:02.624532 31028 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6a649fe6c98494e90c82b8d44568eea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:30:02.624655 31028 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6a649fe6c98494e90c82b8d44568eea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:30:02.624802 31028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6a649fe6c98494e90c82b8d44568eea [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:30:02.626979 31028 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6a649fe6c98494e90c82b8d44568eea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a649fe6c98494e90c82b8d44568eea" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 45491 } }
I20260808 23:30:02.627414 31028 leader_election.cc:302] T 00000000000000000000000000000000 P f6a649fe6c98494e90c82b8d44568eea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6a649fe6c98494e90c82b8d44568eea; no voters:
I20260808 23:30:02.627985 31028 leader_election.cc:288] T 00000000000000000000000000000000 P f6a649fe6c98494e90c82b8d44568eea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:30:02.628082 31035 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6a649fe6c98494e90c82b8d44568eea [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:30:02.628713 31035 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6a649fe6c98494e90c82b8d44568eea [term 1 LEADER]: Becoming Leader. State: Replica: f6a649fe6c98494e90c82b8d44568eea, State: Running, Role: LEADER
I20260808 23:30:02.629208 31035 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6a649fe6c98494e90c82b8d44568eea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a649fe6c98494e90c82b8d44568eea" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 45491 } }
I20260808 23:30:02.629801 31028 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6a649fe6c98494e90c82b8d44568eea [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:30:02.631781 31037 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6a649fe6c98494e90c82b8d44568eea [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6a649fe6c98494e90c82b8d44568eea. Latest consensus state: current_term: 1 leader_uuid: "f6a649fe6c98494e90c82b8d44568eea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a649fe6c98494e90c82b8d44568eea" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 45491 } } }
I20260808 23:30:02.632206 31037 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6a649fe6c98494e90c82b8d44568eea [sys.catalog]: This master's current role is: LEADER
I20260808 23:30:02.633406 31036 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6a649fe6c98494e90c82b8d44568eea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6a649fe6c98494e90c82b8d44568eea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a649fe6c98494e90c82b8d44568eea" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 45491 } } }
I20260808 23:30:02.633688 31036 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6a649fe6c98494e90c82b8d44568eea [sys.catalog]: This master's current role is: LEADER
I20260808 23:30:02.633997 31042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:30:02.638583 31042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:30:02.647171 31042 catalog_manager.cc:1347] Generated new cluster ID: 2e71c63203a6403cbff9a55da8912e91
I20260808 23:30:02.647260 31042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:30:02.663549 31042 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:30:02.664724 31042 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:30:02.676200 31042 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6a649fe6c98494e90c82b8d44568eea: Generated new TSK 0
I20260808 23:30:02.676927 31042 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:30:02.735507 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 30925
I20260808 23:30:02.736618 31020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:30:02.832037 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 30925
2026-08-08T23:30:02Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2090 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:30:02 dist-test-slave-6bbx krb5kdc[31068](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:30:02 dist-test-slave-6bbx krb5kdc[31068](info): set up 2 sockets
Aug 08 23:30:02 dist-test-slave-6bbx krb5kdc[31068](info): commencing operation
krb5kdc: starting...
W20260808 23:30:04.024302 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.008s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:30:04 dist-test-slave-6bbx krb5kdc[31068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:30:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:30:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:30:04.256511 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:37873
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:43633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:37873
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:30:04.575650 31083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:30:04.575938 31083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:30:04.576074 31083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:30:04.582896 31083 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:30:04.582996 31083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:30:04.583076 31083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:30:04.583155 31083 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:30:04.583223 31083 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:30:04.593266 31083 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:43633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:37873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:37873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:30:04.594478 31083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:30:04.595707 31083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:30:04.606495 31093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:04.606588 31090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:04.606563 31091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:30:04.607558 31083 server_base.cc:1034] running on GCE node
I20260808 23:30:04.608315 31083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:30:04.609549 31083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:30:04.610745 31083 hybrid_clock.cc:648] HybridClock initialized: now 1786231804610700 us; error 35 us; skew 500 ppm
I20260808 23:30:04.611052 31083 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:30:04 dist-test-slave-6bbx krb5kdc[31068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231804, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:30:04.617265 31083 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:30:04.618708 31083 webserver.cc:460] Webserver started at http://127.9.44.126:39719/ using document root <none> and password file <none>
I20260808 23:30:04.619380 31083 fs_manager.cc:359] Metadata directory not provided
I20260808 23:30:04.619489 31083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:30:04.619791 31083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:30:04.622582 31083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "150ef7b807d049cf8484b15fdb4b98a2"
format_stamp: "Formatted at 2026-08-08 23:30:04 on dist-test-slave-6bbx"
server_key: "00212782afc4ba3e063c71c271b67ed7"
server_key_iv: "5cfb7381cd0f45dfd5ea7ccfe9149ed4"
server_key_version: "encryptionkey@0"
I20260808 23:30:04.623274 31083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "150ef7b807d049cf8484b15fdb4b98a2"
format_stamp: "Formatted at 2026-08-08 23:30:04 on dist-test-slave-6bbx"
server_key: "00212782afc4ba3e063c71c271b67ed7"
server_key_iv: "5cfb7381cd0f45dfd5ea7ccfe9149ed4"
server_key_version: "encryptionkey@0"
I20260808 23:30:04.628049 31083 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260808 23:30:04.631211 31100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:04.632452 31083 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:30:04.632591 31083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "150ef7b807d049cf8484b15fdb4b98a2"
format_stamp: "Formatted at 2026-08-08 23:30:04 on dist-test-slave-6bbx"
server_key: "00212782afc4ba3e063c71c271b67ed7"
server_key_iv: "5cfb7381cd0f45dfd5ea7ccfe9149ed4"
server_key_version: "encryptionkey@0"
I20260808 23:30:04.632743 31083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:30:04.660905 31083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:30:04.664814 31083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:30:04.665032 31083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:30:04.681723 31083 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:37873
I20260808 23:30:04.681756 31178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:37873 every 8 connection(s)
I20260808 23:30:04.682948 31083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:30:04.686306 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 31083
I20260808 23:30:04.686496 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:30:04.686532 31179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:30:04.687168 9393 external_mini_cluster.cc:1428] Setting key 2a0b0da885ee90142c165be85b9c54fd
I20260808 23:30:04.699455 31179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2: Bootstrap starting.
Aug 08 23:30:04 dist-test-slave-6bbx krb5kdc[31068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:30:04.702992 31179 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2: Neither blocks nor log segments found. Creating new log.
I20260808 23:30:04.704339 31179 log.cc:824] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2: Log is configured to *not* fsync() on all Append() calls
I20260808 23:30:04.707372 31179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2: No bootstrap required, opened a new log
I20260808 23:30:04.710129 31182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:04.689007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41078 (local address 127.9.44.126:37873)
0808 23:30:04.689775 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:04.689786 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:04.689825 (+ 39us) server_negotiation.cc:407] Connection header received
0808 23:30:04.692091 (+ 2266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:04.692112 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:04.692461 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:04.692763 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:04.694455 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:04.695702 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:04.697500 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:04.698109 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:04.698252 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:04.702803 (+ 4551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:04.702815 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:04.702832 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:04.702879 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:04.706306 (+ 3427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:04.706934 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:04.706943 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:04.706950 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:04.707166 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:04.707517 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:04.707521 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:04.707523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:04.708068 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:04.708364 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:04.708822 (+ 458us) server_negotiation.cc:299] Negotiation successful
0808 23:30:04.709208 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":343,"threads_started":1}
I20260808 23:30:04.714939 31179 raft_consensus.cc:348] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150ef7b807d049cf8484b15fdb4b98a2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37873 } }
I20260808 23:30:04.715359 31179 raft_consensus.cc:374] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:30:04.715476 31179 raft_consensus.cc:729] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 150ef7b807d049cf8484b15fdb4b98a2, State: Initialized, Role: FOLLOWER
I20260808 23:30:04.716095 31179 consensus_queue.cc:260] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150ef7b807d049cf8484b15fdb4b98a2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37873 } }
I20260808 23:30:04.716328 31179 raft_consensus.cc:388] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:30:04.716427 31179 raft_consensus.cc:482] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:30:04.716548 31179 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:30:04.718528 31179 raft_consensus.cc:504] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150ef7b807d049cf8484b15fdb4b98a2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37873 } }
I20260808 23:30:04.718957 31179 leader_election.cc:302] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 150ef7b807d049cf8484b15fdb4b98a2; no voters:
I20260808 23:30:04.719478 31179 leader_election.cc:288] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:30:04.719568 31184 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:30:04.721285 31179 sys_catalog.cc:564] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:30:04.721627 31184 raft_consensus.cc:686] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [term 1 LEADER]: Becoming Leader. State: Replica: 150ef7b807d049cf8484b15fdb4b98a2, State: Running, Role: LEADER
I20260808 23:30:04.722067 31184 consensus_queue.cc:237] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150ef7b807d049cf8484b15fdb4b98a2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37873 } }
I20260808 23:30:04.724828 31185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "150ef7b807d049cf8484b15fdb4b98a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150ef7b807d049cf8484b15fdb4b98a2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37873 } } }
I20260808 23:30:04.724804 31186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 150ef7b807d049cf8484b15fdb4b98a2. Latest consensus state: current_term: 1 leader_uuid: "150ef7b807d049cf8484b15fdb4b98a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150ef7b807d049cf8484b15fdb4b98a2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37873 } } }
I20260808 23:30:04.725198 31185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [sys.catalog]: This master's current role is: LEADER
I20260808 23:30:04.725256 31186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [sys.catalog]: This master's current role is: LEADER
I20260808 23:30:04.725929 31197 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:30:04.732652 31197 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:30:04.740949 31197 catalog_manager.cc:1347] Generated new cluster ID: 13cb099ddea348d39906a9fc7685aae3
I20260808 23:30:04.741057 31197 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:30:04.750377 31197 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:30:04.751528 31197 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:30:04.765306 31197 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2: Generated new TSK 0
I20260808 23:30:04.765967 31197 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.44.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
I20260808 23:30:04.840703 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:0
--local_ip_for_outbound_sockets=127.9.44.65
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:37873
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:43633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:30:05.172646 31216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:30:05.172904 31216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:30:05.173034 31216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:30:05.179903 31216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:30:05.179992 31216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:30:05.180132 31216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:30:05.189146 31216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:43633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:37873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:30:05.190253 31216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:30:05.191494 31216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:30:05.203985 31222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:05.203949 31227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:05.204067 31223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:30:05.204644 31216 server_base.cc:1034] running on GCE node
I20260808 23:30:05.205224 31216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:30:05.206085 31216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:30:05.207283 31216 hybrid_clock.cc:648] HybridClock initialized: now 1786231805207229 us; error 45 us; skew 500 ppm
I20260808 23:30:05.207573 31216 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:30:05 dist-test-slave-6bbx krb5kdc[31068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231805, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:30:05.210644 31216 init.cc:377] Logged in from keytab as kudu/127.9.44.65@KRBTEST.COM (short username kudu)
I20260808 23:30:05.214023 31216 webserver.cc:460] Webserver started at http://127.9.44.65:39615/ using document root <none> and password file <none>
I20260808 23:30:05.214646 31216 fs_manager.cc:359] Metadata directory not provided
I20260808 23:30:05.214753 31216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:30:05.215044 31216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:30:05.217782 31216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/data/instance:
uuid: "7d818e1e5401472f9f3b0a667fee06b7"
format_stamp: "Formatted at 2026-08-08 23:30:05 on dist-test-slave-6bbx"
server_key: "773a831111f50dc2339c39bb037c4d3b"
server_key_iv: "9ffb05ece254f5ee1428a5acf459ea66"
server_key_version: "encryptionkey@0"
I20260808 23:30:05.218513 31216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance:
uuid: "7d818e1e5401472f9f3b0a667fee06b7"
format_stamp: "Formatted at 2026-08-08 23:30:05 on dist-test-slave-6bbx"
server_key: "773a831111f50dc2339c39bb037c4d3b"
server_key_iv: "9ffb05ece254f5ee1428a5acf459ea66"
server_key_version: "encryptionkey@0"
I20260808 23:30:05.223438 31216 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 23:30:05.226848 31239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:05.228075 31216 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 23:30:05.228219 31216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "7d818e1e5401472f9f3b0a667fee06b7"
format_stamp: "Formatted at 2026-08-08 23:30:05 on dist-test-slave-6bbx"
server_key: "773a831111f50dc2339c39bb037c4d3b"
server_key_iv: "9ffb05ece254f5ee1428a5acf459ea66"
server_key_version: "encryptionkey@0"
I20260808 23:30:05.228394 31216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:30:05.249538 31216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:30:05.253154 31216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:30:05.253397 31216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:30:05.254292 31216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:30:05.255539 31216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:30:05.255614 31216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:05.255754 31216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:30:05.255806 31216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:05.285683 31216 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:44937
I20260808 23:30:05.285745 31397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:44937 every 8 connection(s)
I20260808 23:30:05.286878 31216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:30:05.295567 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 31216
I20260808 23:30:05.295804 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance
I20260808 23:30:05.296612 9393 external_mini_cluster.cc:1428] Setting key 5d10a93b3bdf27e819b6139129566711
Aug 08 23:30:05 dist-test-slave-6bbx krb5kdc[31068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231805, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:30:05.306161 31401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:05.289242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:49401 (local address 127.9.44.126:37873)
0808 23:30:05.289685 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:05.289691 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:05.290612 (+ 921us) server_negotiation.cc:407] Connection header received
0808 23:30:05.292364 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:05.292369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:05.292436 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:05.292529 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:05.293837 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:05.294558 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:05.296524 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:05.296949 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:05.297049 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:05.300762 (+ 3713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:05.300767 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:05.300770 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:05.300802 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:05.303223 (+ 2421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:05.303963 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:05.303967 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:05.303969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:05.304097 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:05.304518 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:05.304522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:05.304524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:05.304767 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:05.304910 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:05.305569 (+ 659us) server_negotiation.cc:299] Negotiation successful
0808 23:30:05.305806 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":198,"threads_started":1}
I20260808 23:30:05.306933 31402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:05.289775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.65:49401)
0808 23:30:05.290455 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:05.290501 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:05.292134 (+ 1633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:05.292716 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:05.292724 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:05.293185 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:30:05.293699 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:05.293727 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:05.294672 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:05.294676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:05.296369 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:05.296384 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:05.297172 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:05.297179 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:05.297367 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:05.297961 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:30:05.297987 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:30:05.300556 (+ 2569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:30:05.303364 (+ 2808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:05.303374 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:05.303388 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:05.303843 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:05.304202 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:05.304207 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:05.304211 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:05.304403 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:05.304893 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:30:05.304900 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:30:05.305196 (+ 296us) client_negotiation.cc:769] Sending connection context
0808 23:30:05.305425 (+ 229us) client_negotiation.cc:240] Negotiation successful
0808 23:30:05.305880 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":260,"threads_started":1}
I20260808 23:30:05.308291 31398 heartbeater.cc:347] Connected to a master server at 127.9.44.126:37873
I20260808 23:30:05.308595 31398 heartbeater.cc:464] Registering TS with master...
I20260808 23:30:05.309365 31398 heartbeater.cc:511] Master 127.9.44.126:37873 requested a full tablet report, sending...
I20260808 23:30:05.311455 31129 ts_manager.cc:194] Registered new tserver with Master: 7d818e1e5401472f9f3b0a667fee06b7 (127.9.44.65:44937)
I20260808 23:30:05.313383 31129 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.65@KRBTEST.COM'} at 127.9.44.65:49401
WARNING: no policy specified for kudu/127.9.44.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
I20260808 23:30:05.354189 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:0
--local_ip_for_outbound_sockets=127.9.44.66
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:37873
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:43633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:30:05.668049 31405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:30:05.668298 31405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:30:05.668395 31405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:30:05.675298 31405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:30:05.675390 31405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:30:05.675532 31405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:30:05.684723 31405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:43633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:37873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:30:05.685787 31405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:30:05.686949 31405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:30:05.699594 31417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:05.699738 31418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:05.699922 31421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:30:05.700934 31405 server_base.cc:1034] running on GCE node
I20260808 23:30:05.701489 31405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:30:05.702354 31405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:30:05.703560 31405 hybrid_clock.cc:648] HybridClock initialized: now 1786231805703501 us; error 46 us; skew 500 ppm
I20260808 23:30:05.703873 31405 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:30:05 dist-test-slave-6bbx krb5kdc[31068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231805, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:30:05.706792 31405 init.cc:377] Logged in from keytab as kudu/127.9.44.66@KRBTEST.COM (short username kudu)
I20260808 23:30:05.708106 31405 webserver.cc:460] Webserver started at http://127.9.44.66:46007/ using document root <none> and password file <none>
I20260808 23:30:05.708739 31405 fs_manager.cc:359] Metadata directory not provided
I20260808 23:30:05.708837 31405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:30:05.709080 31405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:30:05.711788 31405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/data/instance:
uuid: "06d7d5fb6584491faf26ab94a4341c39"
format_stamp: "Formatted at 2026-08-08 23:30:05 on dist-test-slave-6bbx"
server_key: "b08e2ace371eb613a9f211cb457242be"
server_key_iv: "91e68521b0fccdbd05ed51302fa941f1"
server_key_version: "encryptionkey@0"
I20260808 23:30:05.712531 31405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance:
uuid: "06d7d5fb6584491faf26ab94a4341c39"
format_stamp: "Formatted at 2026-08-08 23:30:05 on dist-test-slave-6bbx"
server_key: "b08e2ace371eb613a9f211cb457242be"
server_key_iv: "91e68521b0fccdbd05ed51302fa941f1"
server_key_version: "encryptionkey@0"
I20260808 23:30:05.717162 31405 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 23:30:05.720304 31430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:05.721503 31405 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 23:30:05.721653 31405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "06d7d5fb6584491faf26ab94a4341c39"
format_stamp: "Formatted at 2026-08-08 23:30:05 on dist-test-slave-6bbx"
server_key: "b08e2ace371eb613a9f211cb457242be"
server_key_iv: "91e68521b0fccdbd05ed51302fa941f1"
server_key_version: "encryptionkey@0"
I20260808 23:30:05.721828 31405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:30:05.734683 31405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:30:05.738353 31405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:30:05.738595 31405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:30:05.739410 31405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:30:05.740667 31405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:30:05.740741 31405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:05.740828 31405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:30:05.740869 31405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:05.769778 31405 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:33305
I20260808 23:30:05.769825 31598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:33305 every 8 connection(s)
I20260808 23:30:05.770931 31405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:30:05.774665 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 31405
I20260808 23:30:05.774866 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance
I20260808 23:30:05.775312 9393 external_mini_cluster.cc:1428] Setting key 9aa400e41d349c3983d83be16f586894
Aug 08 23:30:05 dist-test-slave-6bbx krb5kdc[31068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231805, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:30:05.790546 31401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:05.773310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:37469 (local address 127.9.44.126:37873)
0808 23:30:05.773484 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:05.773489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:05.774551 (+ 1062us) server_negotiation.cc:407] Connection header received
0808 23:30:05.776718 (+ 2167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:05.776723 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:05.776788 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:05.776869 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:05.778428 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:05.779417 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:05.780930 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:05.781496 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:05.781615 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:05.784930 (+ 3315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:05.784936 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:05.784940 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:05.784975 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:05.787770 (+ 2795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:05.788455 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:05.788461 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:05.788464 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:05.788624 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:05.789008 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:05.789012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:05.789015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:05.789312 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:05.789447 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:05.790151 (+ 704us) server_negotiation.cc:299] Negotiation successful
0808 23:30:05.790309 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 23:30:05.791771 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:05.773732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.66:37469)
0808 23:30:05.774402 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:05.774440 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:05.776438 (+ 1998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:05.777146 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:05.777157 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:05.777719 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:30:05.778261 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:05.778279 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:05.779542 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:05.779545 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:05.780791 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:05.780804 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:05.781675 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:05.781680 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:05.781884 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:05.782487 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:30:05.782508 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:30:05.784716 (+ 2208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:30:05.787888 (+ 3172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:05.787894 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:05.787904 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:05.788321 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:05.788718 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:05.788722 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:05.788724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:05.788902 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:05.789427 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:30:05.789434 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:30:05.789797 (+ 363us) client_negotiation.cc:769] Sending connection context
0808 23:30:05.790076 (+ 279us) client_negotiation.cc:240] Negotiation successful
0808 23:30:05.790546 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":258,"threads_started":1}
I20260808 23:30:05.793049 31601 heartbeater.cc:347] Connected to a master server at 127.9.44.126:37873
I20260808 23:30:05.793311 31601 heartbeater.cc:464] Registering TS with master...
I20260808 23:30:05.793951 31601 heartbeater.cc:511] Master 127.9.44.126:37873 requested a full tablet report, sending...
I20260808 23:30:05.795325 31129 ts_manager.cc:194] Registered new tserver with Master: 06d7d5fb6584491faf26ab94a4341c39 (127.9.44.66:33305)
I20260808 23:30:05.796370 31129 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.66@KRBTEST.COM'} at 127.9.44.66:37469
WARNING: no policy specified for kudu/127.9.44.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
I20260808 23:30:05.834897 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:0
--local_ip_for_outbound_sockets=127.9.44.67
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:37873
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:43633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:30:06.157256 31607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:30:06.157505 31607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:30:06.157647 31607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:30:06.164772 31607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:30:06.164901 31607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:30:06.165058 31607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:30:06.174239 31607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:43633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:37873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:30:06.175406 31607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:30:06.176724 31607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:30:06.189021 31613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:06.188928 31617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:06.189002 31615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:30:06.189707 31607 server_base.cc:1034] running on GCE node
I20260808 23:30:06.190294 31607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:30:06.191146 31607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:30:06.192337 31607 hybrid_clock.cc:648] HybridClock initialized: now 1786231806192286 us; error 43 us; skew 500 ppm
I20260808 23:30:06.192644 31607 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:30:06 dist-test-slave-6bbx krb5kdc[31068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231806, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:30:06.195597 31607 init.cc:377] Logged in from keytab as kudu/127.9.44.67@KRBTEST.COM (short username kudu)
I20260808 23:30:06.196975 31607 webserver.cc:460] Webserver started at http://127.9.44.67:36225/ using document root <none> and password file <none>
I20260808 23:30:06.197643 31607 fs_manager.cc:359] Metadata directory not provided
I20260808 23:30:06.197762 31607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:30:06.198047 31607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:30:06.200891 31607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/data/instance:
uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec"
format_stamp: "Formatted at 2026-08-08 23:30:06 on dist-test-slave-6bbx"
server_key: "d5f6ccb0e5f9c32337e15084c110d63e"
server_key_iv: "338032937239cb6996982512e618e913"
server_key_version: "encryptionkey@0"
I20260808 23:30:06.201617 31607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance:
uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec"
format_stamp: "Formatted at 2026-08-08 23:30:06 on dist-test-slave-6bbx"
server_key: "d5f6ccb0e5f9c32337e15084c110d63e"
server_key_iv: "338032937239cb6996982512e618e913"
server_key_version: "encryptionkey@0"
I20260808 23:30:06.206496 31607 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260808 23:30:06.209695 31628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:06.210897 31607 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 23:30:06.211051 31607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec"
format_stamp: "Formatted at 2026-08-08 23:30:06 on dist-test-slave-6bbx"
server_key: "d5f6ccb0e5f9c32337e15084c110d63e"
server_key_iv: "338032937239cb6996982512e618e913"
server_key_version: "encryptionkey@0"
I20260808 23:30:06.211226 31607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:30:06.242946 31607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:30:06.246584 31607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:30:06.246855 31607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:30:06.247797 31607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:30:06.249096 31607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:30:06.249177 31607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:06.249310 31607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:30:06.249373 31607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:06.280737 31607 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:44281
I20260808 23:30:06.280786 31783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:44281 every 8 connection(s)
I20260808 23:30:06.281982 31607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:30:06.286970 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 31607
I20260808 23:30:06.287140 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance
I20260808 23:30:06.287565 9393 external_mini_cluster.cc:1428] Setting key ffdce69acfd3e9091dcb7aaeeb3afc14
Aug 08 23:30:06 dist-test-slave-6bbx krb5kdc[31068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231806, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:30:06.301839 31401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:06.284363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:54755 (local address 127.9.44.126:37873)
0808 23:30:06.284520 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:06.284524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:06.285808 (+ 1284us) server_negotiation.cc:407] Connection header received
0808 23:30:06.288389 (+ 2581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:06.288396 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:06.288485 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:06.288591 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:06.290400 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.291343 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.293041 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.293455 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.293568 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:06.296872 (+ 3304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:06.296877 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:06.296880 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:06.296912 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:06.299218 (+ 2306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:06.299884 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:06.299887 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:06.299889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:06.300022 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:06.300348 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:06.300351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:06.300352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:06.300609 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:06.300718 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:06.301317 (+ 599us) server_negotiation.cc:299] Negotiation successful
0808 23:30:06.301520 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 23:30:06.302760 31787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:06.284817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.67:54755)
0808 23:30:06.285613 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:06.285661 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:06.288030 (+ 2369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:06.288873 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:06.288881 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:06.289546 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:30:06.290221 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:06.290241 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.291504 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.291507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:06.292900 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:06.292914 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.293680 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.293688 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:06.293922 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:06.294478 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:30:06.294504 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:30:06.296680 (+ 2176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:30:06.299335 (+ 2655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:06.299341 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:06.299351 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:06.299789 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:06.300100 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:06.300103 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:06.300105 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:06.300270 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:06.300697 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:30:06.300702 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:30:06.300978 (+ 276us) client_negotiation.cc:769] Sending connection context
0808 23:30:06.301180 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 23:30:06.301560 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":332,"threads_started":1}
I20260808 23:30:06.304000 31784 heartbeater.cc:347] Connected to a master server at 127.9.44.126:37873
I20260808 23:30:06.304242 31784 heartbeater.cc:464] Registering TS with master...
I20260808 23:30:06.304867 31784 heartbeater.cc:511] Master 127.9.44.126:37873 requested a full tablet report, sending...
I20260808 23:30:06.306171 31129 ts_manager.cc:194] Registered new tserver with Master: ca66f7e2cb6e47a1a4820f72a41e40ec (127.9.44.67:44281)
I20260808 23:30:06.306932 31129 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.67@KRBTEST.COM'} at 127.9.44.67:54755
I20260808 23:30:06.315096 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:30:06.316627 31398 heartbeater.cc:503] Master 127.9.44.126:37873 was elected leader, sending a full tablet report...
I20260808 23:30:06.331578 31401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:06.319874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41080 (local address 127.9.44.126:37873)
0808 23:30:06.320013 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:06.320017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:06.320452 (+ 435us) server_negotiation.cc:407] Connection header received
0808 23:30:06.320611 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:06.320614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:06.320672 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:06.320750 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:06.322177 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.322989 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.324531 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.324955 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.325049 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:06.327371 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:06.327376 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:06.327379 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:06.327408 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:06.329594 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:06.330149 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:06.330152 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:06.330154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:06.330268 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:06.330526 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:06.330528 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:06.330530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:06.330803 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:06.331005 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:06.331214 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 23:30:06.331363 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 23:30:06.335103 31129 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41080:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:30:06.342231 31129 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:30:06.364895 31800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:06.353598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:44281 (local address 127.0.0.1:42378)
0808 23:30:06.354670 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:06.354725 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:06.354918 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:06.356260 (+ 1342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:06.356268 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:06.356301 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:30:06.356781 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:06.356795 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.359246 (+ 2451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.359251 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:06.361803 (+ 2552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:06.361820 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.363875 (+ 2055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.363882 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:06.364098 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:06.364153 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 23:30:06.364262 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 23:30:06.364473 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":824,"threads_started":1}
I20260808 23:30:06.366514 31799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:06.353910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42378 (local address 127.9.44.67:44281)
0808 23:30:06.355576 (+ 1666us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:06.355585 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:06.355618 (+ 33us) server_negotiation.cc:407] Connection header received
0808 23:30:06.355742 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:06.355750 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:06.356005 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:06.357011 (+ 1006us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:30:06.357288 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.359073 (+ 1785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.362041 (+ 2968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.363581 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.365628 (+ 2047us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:06.365901 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:06.365973 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 23:30:06.366142 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1476,"thread_start_us":1320,"threads_started":1}
I20260808 23:30:06.372025 31696 tablet_service.cc:1467] Processing CreateTablet for tablet 5b3680b60e96425ba9db7c3362dc23e4 (DEFAULT_TABLE table=test-table [id=2ea28262fae648e391b952edf172aa30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:30:06.373482 31696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b3680b60e96425ba9db7c3362dc23e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:30:06.375615 31798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:06.355426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:33305 (local address 127.0.0.1:55990)
0808 23:30:06.367344 (+ 11918us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:06.367366 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:06.367463 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:06.368553 (+ 1090us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:06.368558 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:06.368584 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:30:06.368941 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:06.368955 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.371035 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.371040 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:06.373036 (+ 1996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:06.373056 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.374838 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.374844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:06.375031 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:06.375095 (+ 64us) client_negotiation.cc:769] Sending connection context
0808 23:30:06.375190 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 23:30:06.375310 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11797,"thread_start_us":13949,"threads_started":1}
I20260808 23:30:06.375739 31805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:06.355800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55990 (local address 127.9.44.66:33305)
0808 23:30:06.367974 (+ 12174us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:06.367982 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:06.368008 (+ 26us) server_negotiation.cc:407] Connection header received
0808 23:30:06.368104 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:06.368111 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:06.368337 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:06.369186 (+ 849us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:30:06.369412 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.370874 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.373215 (+ 2341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.374612 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.374735 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:06.374912 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:06.375245 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 23:30:06.375415 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11999,"thread_start_us":11893,"threads_started":1}
I20260808 23:30:06.376410 31800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:06.353998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:44937 (local address 127.0.0.1:34242)
0808 23:30:06.365349 (+ 11351us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:06.365368 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:06.365470 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:06.370530 (+ 5060us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:06.370534 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:06.370559 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:30:06.370782 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:06.370794 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.372419 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.372424 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:06.374061 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:06.374077 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.375802 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.375808 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:06.376002 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:06.376033 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 23:30:06.376092 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 23:30:06.376205 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11228,"thread_start_us":657,"threads_started":1}
I20260808 23:30:06.377362 31803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:06.354508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34242 (local address 127.9.44.65:44937)
0808 23:30:06.370004 (+ 15496us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:06.370011 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:06.370032 (+ 21us) server_negotiation.cc:407] Connection header received
0808 23:30:06.370103 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:06.370109 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:06.370326 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:06.370958 (+ 632us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:30:06.371239 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.372276 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.374247 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.375578 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.376608 (+ 1030us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:06.376841 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:06.376910 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 23:30:06.377062 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15334,"thread_start_us":15175,"threads_started":1}
I20260808 23:30:06.379155 31502 tablet_service.cc:1467] Processing CreateTablet for tablet 5b3680b60e96425ba9db7c3362dc23e4 (DEFAULT_TABLE table=test-table [id=2ea28262fae648e391b952edf172aa30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:30:06.380395 31502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b3680b60e96425ba9db7c3362dc23e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:30:06.380215 31305 tablet_service.cc:1467] Processing CreateTablet for tablet 5b3680b60e96425ba9db7c3362dc23e4 (DEFAULT_TABLE table=test-table [id=2ea28262fae648e391b952edf172aa30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:30:06.381286 31305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b3680b60e96425ba9db7c3362dc23e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:30:06.390364 31806 tablet_bootstrap.cc:492] T 5b3680b60e96425ba9db7c3362dc23e4 P ca66f7e2cb6e47a1a4820f72a41e40ec: Bootstrap starting.
I20260808 23:30:06.393950 31809 tablet_bootstrap.cc:492] T 5b3680b60e96425ba9db7c3362dc23e4 P 7d818e1e5401472f9f3b0a667fee06b7: Bootstrap starting.
I20260808 23:30:06.394050 31806 tablet_bootstrap.cc:654] T 5b3680b60e96425ba9db7c3362dc23e4 P ca66f7e2cb6e47a1a4820f72a41e40ec: Neither blocks nor log segments found. Creating new log.
I20260808 23:30:06.395468 31806 log.cc:824] T 5b3680b60e96425ba9db7c3362dc23e4 P ca66f7e2cb6e47a1a4820f72a41e40ec: Log is configured to *not* fsync() on all Append() calls
I20260808 23:30:06.397274 31809 tablet_bootstrap.cc:654] T 5b3680b60e96425ba9db7c3362dc23e4 P 7d818e1e5401472f9f3b0a667fee06b7: Neither blocks nor log segments found. Creating new log.
I20260808 23:30:06.398365 31809 log.cc:824] T 5b3680b60e96425ba9db7c3362dc23e4 P 7d818e1e5401472f9f3b0a667fee06b7: Log is configured to *not* fsync() on all Append() calls
I20260808 23:30:06.398679 31806 tablet_bootstrap.cc:492] T 5b3680b60e96425ba9db7c3362dc23e4 P ca66f7e2cb6e47a1a4820f72a41e40ec: No bootstrap required, opened a new log
I20260808 23:30:06.398940 31806 ts_tablet_manager.cc:1397] T 5b3680b60e96425ba9db7c3362dc23e4 P ca66f7e2cb6e47a1a4820f72a41e40ec: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260808 23:30:06.399039 31808 tablet_bootstrap.cc:492] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39: Bootstrap starting.
I20260808 23:30:06.401337 31809 tablet_bootstrap.cc:492] T 5b3680b60e96425ba9db7c3362dc23e4 P 7d818e1e5401472f9f3b0a667fee06b7: No bootstrap required, opened a new log
I20260808 23:30:06.401628 31809 ts_tablet_manager.cc:1397] T 5b3680b60e96425ba9db7c3362dc23e4 P 7d818e1e5401472f9f3b0a667fee06b7: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260808 23:30:06.402403 31808 tablet_bootstrap.cc:654] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39: Neither blocks nor log segments found. Creating new log.
I20260808 23:30:06.403683 31808 log.cc:824] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39: Log is configured to *not* fsync() on all Append() calls
I20260808 23:30:06.406702 31806 raft_consensus.cc:348] T 5b3680b60e96425ba9db7c3362dc23e4 P ca66f7e2cb6e47a1a4820f72a41e40ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } } peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } }
I20260808 23:30:06.407037 31806 raft_consensus.cc:374] T 5b3680b60e96425ba9db7c3362dc23e4 P ca66f7e2cb6e47a1a4820f72a41e40ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:30:06.407181 31806 raft_consensus.cc:729] T 5b3680b60e96425ba9db7c3362dc23e4 P ca66f7e2cb6e47a1a4820f72a41e40ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca66f7e2cb6e47a1a4820f72a41e40ec, State: Initialized, Role: FOLLOWER
I20260808 23:30:06.407874 31806 consensus_queue.cc:260] T 5b3680b60e96425ba9db7c3362dc23e4 P ca66f7e2cb6e47a1a4820f72a41e40ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } } peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } }
I20260808 23:30:06.408906 31784 heartbeater.cc:503] Master 127.9.44.126:37873 was elected leader, sending a full tablet report...
I20260808 23:30:06.409514 31808 tablet_bootstrap.cc:492] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39: No bootstrap required, opened a new log
I20260808 23:30:06.409511 31806 ts_tablet_manager.cc:1428] T 5b3680b60e96425ba9db7c3362dc23e4 P ca66f7e2cb6e47a1a4820f72a41e40ec: Time spent starting tablet: real 0.010s user 0.006s sys 0.003s
I20260808 23:30:06.409765 31808 ts_tablet_manager.cc:1397] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260808 23:30:06.412068 31809 raft_consensus.cc:348] T 5b3680b60e96425ba9db7c3362dc23e4 P 7d818e1e5401472f9f3b0a667fee06b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } } peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } }
I20260808 23:30:06.412479 31809 raft_consensus.cc:374] T 5b3680b60e96425ba9db7c3362dc23e4 P 7d818e1e5401472f9f3b0a667fee06b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:30:06.412578 31809 raft_consensus.cc:729] T 5b3680b60e96425ba9db7c3362dc23e4 P 7d818e1e5401472f9f3b0a667fee06b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d818e1e5401472f9f3b0a667fee06b7, State: Initialized, Role: FOLLOWER
I20260808 23:30:06.413143 31809 consensus_queue.cc:260] T 5b3680b60e96425ba9db7c3362dc23e4 P 7d818e1e5401472f9f3b0a667fee06b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } } peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } }
I20260808 23:30:06.414829 31809 ts_tablet_manager.cc:1428] T 5b3680b60e96425ba9db7c3362dc23e4 P 7d818e1e5401472f9f3b0a667fee06b7: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260808 23:30:06.418795 31808 raft_consensus.cc:348] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } } peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } }
I20260808 23:30:06.419133 31808 raft_consensus.cc:374] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:30:06.419225 31808 raft_consensus.cc:729] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d7d5fb6584491faf26ab94a4341c39, State: Initialized, Role: FOLLOWER
I20260808 23:30:06.419765 31808 consensus_queue.cc:260] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } } peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } }
I20260808 23:30:06.420723 31601 heartbeater.cc:503] Master 127.9.44.126:37873 was elected leader, sending a full tablet report...
I20260808 23:30:06.421361 31808 ts_tablet_manager.cc:1428] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
W20260808 23:30:06.522837 31602 tablet.cc:2369] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:30:06.533540 31785 tablet.cc:2369] T 5b3680b60e96425ba9db7c3362dc23e4 P ca66f7e2cb6e47a1a4820f72a41e40ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:30:06.539100 31400 tablet.cc:2369] T 5b3680b60e96425ba9db7c3362dc23e4 P 7d818e1e5401472f9f3b0a667fee06b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:30:06.699707 31818 raft_consensus.cc:482] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:30:06.700001 31818 raft_consensus.cc:504] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } } peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } }
I20260808 23:30:06.701647 31818 leader_election.cc:288] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca66f7e2cb6e47a1a4820f72a41e40ec (127.9.44.67:44281), 7d818e1e5401472f9f3b0a667fee06b7 (127.9.44.65:44937)
I20260808 23:30:06.708423 31803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:06.702551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:60841 (local address 127.9.44.65:44937)
0808 23:30:06.702752 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:06.702758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:06.703031 (+ 273us) server_negotiation.cc:407] Connection header received
0808 23:30:06.703193 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:06.703196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:06.703263 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:06.703335 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:30:06.704101 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.704783 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.706571 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.707389 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.707512 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:06.707616 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:06.708050 (+ 434us) server_negotiation.cc:299] Negotiation successful
0808 23:30:06.708178 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 23:30:06.708532 31799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:06.702054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:40191 (local address 127.9.44.67:44281)
0808 23:30:06.702300 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:06.702306 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:06.702747 (+ 441us) server_negotiation.cc:407] Connection header received
0808 23:30:06.703068 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:06.703072 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:06.703136 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:06.703215 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:30:06.703828 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.704656 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.706584 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.707517 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.707647 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:06.707790 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:06.708169 (+ 379us) server_negotiation.cc:299] Negotiation successful
0808 23:30:06.708294 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 23:30:06.708576 31820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:06.702092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:44937 (local address 127.9.44.66:60841)
0808 23:30:06.702922 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:06.702936 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:06.703064 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:06.703370 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:06.703374 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:06.703418 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:30:06.703908 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:06.703921 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.704935 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.704941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:06.706420 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:06.706433 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.707617 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.707623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:06.707887 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:06.707913 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 23:30:06.708002 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 23:30:06.708121 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":735,"mutex_wait_us":44,"thread_start_us":509,"threads_started":1}
I20260808 23:30:06.708662 31819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:06.701857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:44281 (local address 127.9.44.66:40191)
0808 23:30:06.702614 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:06.702637 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:06.702900 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:06.703243 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:06.703248 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:06.703275 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:30:06.703644 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:06.703654 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.704789 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.704794 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:06.706424 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:06.706437 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.707809 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.707816 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:06.707987 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:06.708010 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 23:30:06.708090 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 23:30:06.708210 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"mutex_wait_us":106,"thread_start_us":372,"threads_started":1}
I20260808 23:30:06.709712 31727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b3680b60e96425ba9db7c3362dc23e4" candidate_uuid: "06d7d5fb6584491faf26ab94a4341c39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" is_pre_election: true
I20260808 23:30:06.709823 31331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b3680b60e96425ba9db7c3362dc23e4" candidate_uuid: "06d7d5fb6584491faf26ab94a4341c39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d818e1e5401472f9f3b0a667fee06b7" is_pre_election: true
I20260808 23:30:06.710294 31727 raft_consensus.cc:2454] T 5b3680b60e96425ba9db7c3362dc23e4 P ca66f7e2cb6e47a1a4820f72a41e40ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d7d5fb6584491faf26ab94a4341c39 in term 0.
I20260808 23:30:06.710330 31331 raft_consensus.cc:2454] T 5b3680b60e96425ba9db7c3362dc23e4 P 7d818e1e5401472f9f3b0a667fee06b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d7d5fb6584491faf26ab94a4341c39 in term 0.
I20260808 23:30:06.710992 31432 leader_election.cc:302] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d7d5fb6584491faf26ab94a4341c39, ca66f7e2cb6e47a1a4820f72a41e40ec; no voters:
I20260808 23:30:06.711422 31818 raft_consensus.cc:2781] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:30:06.711562 31818 raft_consensus.cc:482] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:30:06.711681 31818 raft_consensus.cc:3037] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:30:06.714208 31818 raft_consensus.cc:504] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } } peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } }
I20260808 23:30:06.714792 31818 leader_election.cc:288] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39 [CANDIDATE]: Term 1 election: Requested vote from peers ca66f7e2cb6e47a1a4820f72a41e40ec (127.9.44.67:44281), 7d818e1e5401472f9f3b0a667fee06b7 (127.9.44.65:44937)
I20260808 23:30:06.715148 31727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b3680b60e96425ba9db7c3362dc23e4" candidate_uuid: "06d7d5fb6584491faf26ab94a4341c39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec"
I20260808 23:30:06.715204 31331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b3680b60e96425ba9db7c3362dc23e4" candidate_uuid: "06d7d5fb6584491faf26ab94a4341c39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d818e1e5401472f9f3b0a667fee06b7"
I20260808 23:30:06.715322 31727 raft_consensus.cc:3037] T 5b3680b60e96425ba9db7c3362dc23e4 P ca66f7e2cb6e47a1a4820f72a41e40ec [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:30:06.715394 31331 raft_consensus.cc:3037] T 5b3680b60e96425ba9db7c3362dc23e4 P 7d818e1e5401472f9f3b0a667fee06b7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:30:06.717489 31727 raft_consensus.cc:2454] T 5b3680b60e96425ba9db7c3362dc23e4 P ca66f7e2cb6e47a1a4820f72a41e40ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d7d5fb6584491faf26ab94a4341c39 in term 1.
I20260808 23:30:06.717864 31432 leader_election.cc:302] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d7d5fb6584491faf26ab94a4341c39, ca66f7e2cb6e47a1a4820f72a41e40ec; no voters:
I20260808 23:30:06.717896 31331 raft_consensus.cc:2454] T 5b3680b60e96425ba9db7c3362dc23e4 P 7d818e1e5401472f9f3b0a667fee06b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d7d5fb6584491faf26ab94a4341c39 in term 1.
I20260808 23:30:06.718117 31818 raft_consensus.cc:2781] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:30:06.718515 31818 raft_consensus.cc:686] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39 [term 1 LEADER]: Becoming Leader. State: Replica: 06d7d5fb6584491faf26ab94a4341c39, State: Running, Role: LEADER
I20260808 23:30:06.718943 31818 consensus_queue.cc:237] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } } peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } }
I20260808 23:30:06.723528 31129 catalog_manager.cc:5495] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06d7d5fb6584491faf26ab94a4341c39 (127.9.44.66). New cstate: current_term: 1 leader_uuid: "06d7d5fb6584491faf26ab94a4341c39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } health_report { overall_health: HEALTHY } } }
I20260808 23:30:06.826038 31805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:06.820695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56006 (local address 127.9.44.66:33305)
0808 23:30:06.820922 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:06.820927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:06.820946 (+ 19us) server_negotiation.cc:407] Connection header received
0808 23:30:06.821044 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:06.821049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:06.821131 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:06.821278 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:30:06.821680 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.822487 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.823913 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.824359 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.824467 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:06.824971 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:30:06.825436 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:30:06.825574 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:06.825654 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 23:30:06.825755 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 23:30:06.834008 31727 raft_consensus.cc:1261] T 5b3680b60e96425ba9db7c3362dc23e4 P ca66f7e2cb6e47a1a4820f72a41e40ec [term 1 FOLLOWER]: Refusing update from remote peer 06d7d5fb6584491faf26ab94a4341c39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:30:06.834007 31331 raft_consensus.cc:1261] T 5b3680b60e96425ba9db7c3362dc23e4 P 7d818e1e5401472f9f3b0a667fee06b7 [term 1 FOLLOWER]: Refusing update from remote peer 06d7d5fb6584491faf26ab94a4341c39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:30:06.834930 31818 consensus_queue.cc:1035] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:30:06.835206 31821 consensus_queue.cc:1035] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:30:06.849763 31824 mvcc.cc:205] Tried to move back new op lower bound from 7316405480782053376 to 7316405480324300800. Current Snapshot: MvccSnapshot[applied={T|T < 7316405480782053376}]
I20260808 23:30:06.852555 31825 mvcc.cc:205] Tried to move back new op lower bound from 7316405480782053376 to 7316405480324300800. Current Snapshot: MvccSnapshot[applied={T|T < 7316405480782053376}]
I20260808 23:30:06.852957 31827 mvcc.cc:205] Tried to move back new op lower bound from 7316405480782053376 to 7316405480324300800. Current Snapshot: MvccSnapshot[applied={T|T < 7316405480782053376}]
I20260808 23:30:06.865926 31129 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:30:06.866145 31129 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:30:06.869462 31129 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2: Sending DeleteTablet for 3 replicas of tablet 5b3680b60e96425ba9db7c3362dc23e4
I20260808 23:30:06.870582 31696 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b3680b60e96425ba9db7c3362dc23e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:30:06 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:42378
I20260808 23:30:06.870630 31305 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b3680b60e96425ba9db7c3362dc23e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:30:06 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:34242
I20260808 23:30:06.870718 31502 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b3680b60e96425ba9db7c3362dc23e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:30:06 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:55990
I20260808 23:30:06.871718 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 31216
I20260808 23:30:06.883081 31842 tablet_replica.cc:330] T 5b3680b60e96425ba9db7c3362dc23e4 P 7d818e1e5401472f9f3b0a667fee06b7: stopping tablet replica
I20260808 23:30:06.883646 31842 raft_consensus.cc:2229] T 5b3680b60e96425ba9db7c3362dc23e4 P 7d818e1e5401472f9f3b0a667fee06b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:30:06.884128 31841 tablet_replica.cc:330] T 5b3680b60e96425ba9db7c3362dc23e4 P ca66f7e2cb6e47a1a4820f72a41e40ec: stopping tablet replica
I20260808 23:30:06.884359 31842 raft_consensus.cc:2258] T 5b3680b60e96425ba9db7c3362dc23e4 P 7d818e1e5401472f9f3b0a667fee06b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:30:06.884547 31841 raft_consensus.cc:2229] T 5b3680b60e96425ba9db7c3362dc23e4 P ca66f7e2cb6e47a1a4820f72a41e40ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:30:06.885010 31841 raft_consensus.cc:2258] T 5b3680b60e96425ba9db7c3362dc23e4 P ca66f7e2cb6e47a1a4820f72a41e40ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:30:06.885392 31844 tablet_replica.cc:330] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39: stopping tablet replica
I20260808 23:30:06.885833 31844 raft_consensus.cc:2229] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:30:06.886611 31844 raft_consensus.cc:2258] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:30:06.887120 31841 ts_tablet_manager.cc:1905] T 5b3680b60e96425ba9db7c3362dc23e4 P ca66f7e2cb6e47a1a4820f72a41e40ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:30:06.888721 31844 ts_tablet_manager.cc:1905] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:30:06.893909 31841 ts_tablet_manager.cc:1918] T 5b3680b60e96425ba9db7c3362dc23e4 P ca66f7e2cb6e47a1a4820f72a41e40ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:30:06.894073 31841 log.cc:1196] T 5b3680b60e96425ba9db7c3362dc23e4 P ca66f7e2cb6e47a1a4820f72a41e40ec: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/5b3680b60e96425ba9db7c3362dc23e4
I20260808 23:30:06.894613 31841 ts_tablet_manager.cc:1939] T 5b3680b60e96425ba9db7c3362dc23e4 P ca66f7e2cb6e47a1a4820f72a41e40ec: Deleting consensus metadata
I20260808 23:30:06.896332 31102 catalog_manager.cc:4841] TS ca66f7e2cb6e47a1a4820f72a41e40ec (127.9.44.67:44281): tablet 5b3680b60e96425ba9db7c3362dc23e4 (table test-table [id=2ea28262fae648e391b952edf172aa30]) successfully deleted
I20260808 23:30:06.896570 31844 ts_tablet_manager.cc:1918] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:30:06.896792 31844 log.cc:1196] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/5b3680b60e96425ba9db7c3362dc23e4
I20260808 23:30:06.896904 31842 ts_tablet_manager.cc:1905] T 5b3680b60e96425ba9db7c3362dc23e4 P 7d818e1e5401472f9f3b0a667fee06b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:30:06.897452 31844 ts_tablet_manager.cc:1939] T 5b3680b60e96425ba9db7c3362dc23e4 P 06d7d5fb6584491faf26ab94a4341c39: Deleting consensus metadata
I20260808 23:30:06.899154 31101 catalog_manager.cc:4841] TS 06d7d5fb6584491faf26ab94a4341c39 (127.9.44.66:33305): tablet 5b3680b60e96425ba9db7c3362dc23e4 (table test-table [id=2ea28262fae648e391b952edf172aa30]) successfully deleted
Aug 08 23:30:06 dist-test-slave-6bbx krb5kdc[31068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.65@KRBTEST.COM
I20260808 23:30:06.903074 31842 ts_tablet_manager.cc:1918] T 5b3680b60e96425ba9db7c3362dc23e4 P 7d818e1e5401472f9f3b0a667fee06b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:30:06.903246 31842 log.cc:1196] T 5b3680b60e96425ba9db7c3362dc23e4 P 7d818e1e5401472f9f3b0a667fee06b7: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/wal/wals/5b3680b60e96425ba9db7c3362dc23e4
I20260808 23:30:06.903798 31842 ts_tablet_manager.cc:1939] T 5b3680b60e96425ba9db7c3362dc23e4 P 7d818e1e5401472f9f3b0a667fee06b7: Deleting consensus metadata
I20260808 23:30:06.905418 31101 catalog_manager.cc:4841] TS 7d818e1e5401472f9f3b0a667fee06b7 (127.9.44.65:44937): tablet 5b3680b60e96425ba9db7c3362dc23e4 (table test-table [id=2ea28262fae648e391b952edf172aa30]) successfully deleted
I20260808 23:30:06.908605 31803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:06.872881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34250 (local address 127.9.44.65:44937)
0808 23:30:06.873042 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:06.873048 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:06.889700 (+ 16652us) server_negotiation.cc:407] Connection header received
0808 23:30:06.890164 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:06.890170 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:06.890253 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:06.890335 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:06.892473 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.893581 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.895688 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:06.896310 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:06.896407 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:06.902057 (+ 5650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:06.902070 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:06.902086 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:06.902143 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:06.905917 (+ 3774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:06.906717 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:06.906722 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:06.906725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:06.906849 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:06.907248 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:06.907251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:06.907253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:06.907581 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:06.907850 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:06.908180 (+ 330us) server_negotiation.cc:299] Negotiation successful
0808 23:30:06.908330 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 23:30:06.909241 31392 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:30:06.995790 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 31216
I20260808 23:30:07.017186 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 31405
Aug 08 23:30:07 dist-test-slave-6bbx krb5kdc[31068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.66@KRBTEST.COM
I20260808 23:30:07.034747 31805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:07.017994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56012 (local address 127.9.44.66:33305)
0808 23:30:07.018196 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:07.018200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:07.018219 (+ 19us) server_negotiation.cc:407] Connection header received
0808 23:30:07.018392 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:07.018396 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:07.018484 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:07.018569 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:07.020251 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:07.021079 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:07.023028 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:07.023499 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:07.023612 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:07.028924 (+ 5312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:07.028937 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:07.028952 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:07.028992 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:07.031970 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:07.032760 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:07.032767 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:07.032769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:07.032907 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:07.033271 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:07.033274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:07.033276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:07.033602 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:07.033877 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:07.034146 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 23:30:07.034311 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 23:30:07.035449 31592 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:30:07.147764 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 31405
I20260808 23:30:07.172140 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 31607
Aug 08 23:30:07 dist-test-slave-6bbx krb5kdc[31068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.67@KRBTEST.COM
I20260808 23:30:07.186697 31799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:07.172853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42394 (local address 127.9.44.67:44281)
0808 23:30:07.173022 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:07.173029 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:07.173086 (+ 57us) server_negotiation.cc:407] Connection header received
0808 23:30:07.173290 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:07.173294 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:07.173376 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:07.173461 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:07.174746 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:07.175452 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:07.176813 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:07.177196 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:07.177316 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:07.181467 (+ 4151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:07.181480 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:07.181495 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:07.181529 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:07.184053 (+ 2524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:07.184694 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:07.184701 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:07.184703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:07.184853 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:07.185202 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:07.185224 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:07.185226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:07.185572 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:07.185845 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:07.186165 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 23:30:07.186383 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 23:30:07.187289 31777 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:30:07.274448 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 31607
I20260808 23:30:07.297135 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 31083
I20260808 23:30:07.298278 31172 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:30:07.406803 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 31083
I20260808 23:30:07.435144 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:37873
--webserver_interface=127.9.44.126
--webserver_port=39719
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:43633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:37873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:30:07.763626 31853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:30:07.763950 31853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:30:07.764091 31853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:30:07.770992 31853 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:30:07.771095 31853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:30:07.771173 31853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:30:07.771245 31853 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:30:07.771306 31853 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:30:07.780709 31853 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:43633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:37873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:37873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=39719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:30:07.781853 31853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:30:07.783099 31853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:30:07.806085 31865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:07.806245 31863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:07.806241 31862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:30:07.807085 31853 server_base.cc:1034] running on GCE node
I20260808 23:30:07.807904 31853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:30:07.809367 31853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:30:07.810592 31853 hybrid_clock.cc:648] HybridClock initialized: now 1786231807810532 us; error 49 us; skew 500 ppm
I20260808 23:30:07.810949 31853 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:30:07 dist-test-slave-6bbx krb5kdc[31068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231807, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:30:07.814265 31853 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:30:07.815819 31853 webserver.cc:460] Webserver started at http://127.9.44.126:39719/ using document root <none> and password file <none>
I20260808 23:30:07.816542 31853 fs_manager.cc:359] Metadata directory not provided
I20260808 23:30:07.816679 31853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:30:07.822310 31853 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.006s
I20260808 23:30:07.825157 31875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:07.826351 31853 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260808 23:30:07.826508 31853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "150ef7b807d049cf8484b15fdb4b98a2"
format_stamp: "Formatted at 2026-08-08 23:30:04 on dist-test-slave-6bbx"
server_key: "00212782afc4ba3e063c71c271b67ed7"
server_key_iv: "5cfb7381cd0f45dfd5ea7ccfe9149ed4"
server_key_version: "encryptionkey@0"
I20260808 23:30:07.827503 31853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:30:07.849722 31853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:30:07.853827 31853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:30:07.854086 31853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:30:07.871582 31853 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:37873
I20260808 23:30:07.871599 31949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:37873 every 8 connection(s)
I20260808 23:30:07.873023 31853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:30:07.879587 31951 sys_catalog.cc:263] Verifying existing consensus state
I20260808 23:30:07.880554 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 31853
I20260808 23:30:07.881489 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:44937
--local_ip_for_outbound_sockets=127.9.44.65
--tserver_master_addrs=127.9.44.126:37873
--webserver_port=39615
--webserver_interface=127.9.44.65
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:43633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:30:07.884061 31951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2: Bootstrap starting.
I20260808 23:30:07.905002 31951 log.cc:824] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2: Log is configured to *not* fsync() on all Append() calls
I20260808 23:30:07.916961 31951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:30:07.917456 31951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2: Bootstrap complete.
I20260808 23:30:07.927747 31951 raft_consensus.cc:348] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150ef7b807d049cf8484b15fdb4b98a2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37873 } }
I20260808 23:30:07.928545 31951 raft_consensus.cc:729] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 150ef7b807d049cf8484b15fdb4b98a2, State: Initialized, Role: FOLLOWER
I20260808 23:30:07.929126 31951 consensus_queue.cc:260] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150ef7b807d049cf8484b15fdb4b98a2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37873 } }
I20260808 23:30:07.929361 31951 raft_consensus.cc:388] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:30:07.929474 31951 raft_consensus.cc:482] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:30:07.929632 31951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [term 1 FOLLOWER]: Advancing to term 2
I20260808 23:30:07.933418 31951 raft_consensus.cc:504] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150ef7b807d049cf8484b15fdb4b98a2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37873 } }
I20260808 23:30:07.933923 31951 leader_election.cc:302] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 150ef7b807d049cf8484b15fdb4b98a2; no voters:
I20260808 23:30:07.934508 31951 leader_election.cc:288] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 23:30:07.934638 31955 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 23:30:07.935899 31955 raft_consensus.cc:686] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [term 2 LEADER]: Becoming Leader. State: Replica: 150ef7b807d049cf8484b15fdb4b98a2, State: Running, Role: LEADER
I20260808 23:30:07.936506 31955 consensus_queue.cc:237] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150ef7b807d049cf8484b15fdb4b98a2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37873 } }
I20260808 23:30:07.936764 31951 sys_catalog.cc:564] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:30:07.939127 31957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 150ef7b807d049cf8484b15fdb4b98a2. Latest consensus state: current_term: 2 leader_uuid: "150ef7b807d049cf8484b15fdb4b98a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150ef7b807d049cf8484b15fdb4b98a2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37873 } } }
I20260808 23:30:07.939546 31957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [sys.catalog]: This master's current role is: LEADER
I20260808 23:30:07.940133 31956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "150ef7b807d049cf8484b15fdb4b98a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150ef7b807d049cf8484b15fdb4b98a2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37873 } } }
I20260808 23:30:07.940454 31956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [sys.catalog]: This master's current role is: LEADER
I20260808 23:30:07.948765 31963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:30:07.957103 31963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:30:07.958510 31963 catalog_manager.cc:1259] Loaded cluster ID: 13cb099ddea348d39906a9fc7685aae3
I20260808 23:30:07.958659 31963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:30:07.961949 31963 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 23:30:07.979790 31963 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 23:30:07.993650 31963 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:30:07.995245 31963 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2: Loaded TSK: 0
I20260808 23:30:07.996778 31963 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 23:30:08.226608 31953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:30:08.226858 31953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:30:08.226971 31953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:30:08.234174 31953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:30:08.234273 31953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:30:08.234405 31953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:30:08.243702 31953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:43633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:44937
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=39615
--tserver_master_addrs=127.9.44.126:37873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:30:08.244848 31953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:30:08.246102 31953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:30:08.258442 31982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:08.258478 31983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:08.259058 31987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:30:08.259687 31953 server_base.cc:1034] running on GCE node
I20260808 23:30:08.260349 31953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:30:08.261348 31953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:30:08.262569 31953 hybrid_clock.cc:648] HybridClock initialized: now 1786231808262519 us; error 40 us; skew 500 ppm
I20260808 23:30:08.262907 31953 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:30:08 dist-test-slave-6bbx krb5kdc[31068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231808, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:30:08.266563 31953 init.cc:377] Logged in from keytab as kudu/127.9.44.65@KRBTEST.COM (short username kudu)
I20260808 23:30:08.268172 31953 webserver.cc:460] Webserver started at http://127.9.44.65:39615/ using document root <none> and password file <none>
I20260808 23:30:08.268968 31953 fs_manager.cc:359] Metadata directory not provided
I20260808 23:30:08.269111 31953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:30:08.275660 31953 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 23:30:08.278775 31994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:08.280341 31953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:30:08.280511 31953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "7d818e1e5401472f9f3b0a667fee06b7"
format_stamp: "Formatted at 2026-08-08 23:30:05 on dist-test-slave-6bbx"
server_key: "773a831111f50dc2339c39bb037c4d3b"
server_key_iv: "9ffb05ece254f5ee1428a5acf459ea66"
server_key_version: "encryptionkey@0"
I20260808 23:30:08.281860 31953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:30:08.301640 31953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:30:08.306252 31953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:30:08.306514 31953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:30:08.307447 31953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:30:08.308774 31953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:30:08.308890 31953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:08.309000 31953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:30:08.309072 31953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:08.336772 31953 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:44937
I20260808 23:30:08.336859 32145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:44937 every 8 connection(s)
I20260808 23:30:08.338222 31953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:30:08.341300 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 31953
I20260808 23:30:08.341965 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:33305
--local_ip_for_outbound_sockets=127.9.44.66
--tserver_master_addrs=127.9.44.126:37873
--webserver_port=46007
--webserver_interface=127.9.44.66
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:43633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:30:08 dist-test-slave-6bbx krb5kdc[31068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231808, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:30:08.362360 32149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:08.341211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.65:59053)
0808 23:30:08.342065 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:08.342131 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:08.343906 (+ 1775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:08.345431 (+ 1525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:08.345441 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:08.346090 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:30:08.346802 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:08.346828 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:08.348798 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:08.348803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:08.350467 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:08.350482 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:08.351626 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:08.351633 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:08.351937 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:08.352609 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:30:08.352636 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:30:08.354964 (+ 2328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:30:08.358335 (+ 3371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:08.358344 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:08.358358 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:08.358749 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:08.359157 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:08.359162 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:08.359166 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:08.359343 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:08.360057 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:30:08.360064 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:30:08.360377 (+ 313us) client_negotiation.cc:769] Sending connection context
0808 23:30:08.360599 (+ 222us) client_negotiation.cc:240] Negotiation successful
0808 23:30:08.361115 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":357,"threads_started":1}
I20260808 23:30:08.362365 32148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:08.340960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:59053 (local address 127.9.44.126:37873)
0808 23:30:08.341565 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:08.341580 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:08.344226 (+ 2646us) server_negotiation.cc:407] Connection header received
0808 23:30:08.344544 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:08.344558 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:08.344913 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:08.345225 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:08.347034 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:08.348623 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:08.350588 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:08.351409 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:08.351560 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:08.355225 (+ 3665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:08.355233 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:08.355246 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:08.355281 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:08.358205 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:08.358875 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:08.358882 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:08.358888 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:08.359046 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:08.359447 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:08.359450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:08.359452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:08.359923 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:08.360106 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:08.360823 (+ 717us) server_negotiation.cc:299] Negotiation successful
0808 23:30:08.361193 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":244,"threads_started":1}
I20260808 23:30:08.365311 32146 heartbeater.cc:347] Connected to a master server at 127.9.44.126:37873
I20260808 23:30:08.365574 32146 heartbeater.cc:464] Registering TS with master...
I20260808 23:30:08.366261 32146 heartbeater.cc:511] Master 127.9.44.126:37873 requested a full tablet report, sending...
I20260808 23:30:08.368696 31899 ts_manager.cc:194] Registered new tserver with Master: 7d818e1e5401472f9f3b0a667fee06b7 (127.9.44.65:44937)
I20260808 23:30:08.370903 31899 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.65@KRBTEST.COM'} at 127.9.44.65:59053
W20260808 23:30:08.671315 32150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:30:08.671555 32150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:30:08.671649 32150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:30:08.678628 32150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:30:08.678716 32150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:30:08.678824 32150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:30:08.688091 32150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:43633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:33305
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=46007
--tserver_master_addrs=127.9.44.126:37873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:30:08.689217 32150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:30:08.690425 32150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:30:08.702534 32155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:08.702715 32157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:08.702721 32160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:30:08.703455 32150 server_base.cc:1034] running on GCE node
I20260808 23:30:08.704066 32150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:30:08.704895 32150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:30:08.706104 32150 hybrid_clock.cc:648] HybridClock initialized: now 1786231808706062 us; error 32 us; skew 500 ppm
I20260808 23:30:08.706413 32150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:30:08 dist-test-slave-6bbx krb5kdc[31068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231808, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:30:08.709412 32150 init.cc:377] Logged in from keytab as kudu/127.9.44.66@KRBTEST.COM (short username kudu)
I20260808 23:30:08.710739 32150 webserver.cc:460] Webserver started at http://127.9.44.66:46007/ using document root <none> and password file <none>
I20260808 23:30:08.711347 32150 fs_manager.cc:359] Metadata directory not provided
I20260808 23:30:08.711457 32150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:30:08.716373 32150 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:30:08.718909 32171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:08.720152 32150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:30:08.720297 32150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "06d7d5fb6584491faf26ab94a4341c39"
format_stamp: "Formatted at 2026-08-08 23:30:05 on dist-test-slave-6bbx"
server_key: "b08e2ace371eb613a9f211cb457242be"
server_key_iv: "91e68521b0fccdbd05ed51302fa941f1"
server_key_version: "encryptionkey@0"
I20260808 23:30:08.721218 32150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:30:08.732894 32150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:30:08.736704 32150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:30:08.736935 32150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:30:08.737808 32150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:30:08.739014 32150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:30:08.739112 32150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:08.739226 32150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:30:08.739300 32150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:08.768710 32150 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:33305
I20260808 23:30:08.768784 32331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:33305 every 8 connection(s)
I20260808 23:30:08.770037 32150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:30:08.779732 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 32150
I20260808 23:30:08.780325 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:44281
--local_ip_for_outbound_sockets=127.9.44.67
--tserver_master_addrs=127.9.44.126:37873
--webserver_port=36225
--webserver_interface=127.9.44.67
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:43633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:30:08 dist-test-slave-6bbx krb5kdc[31068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231808, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:30:08.788796 32148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:08.772305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:34581 (local address 127.9.44.126:37873)
0808 23:30:08.772466 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:08.772471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:08.773541 (+ 1070us) server_negotiation.cc:407] Connection header received
0808 23:30:08.775268 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:08.775274 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:08.775356 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:08.775445 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:08.776900 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:08.777586 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:08.779077 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:08.779641 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:08.779796 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:08.782957 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:08.782963 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:08.782967 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:08.783005 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:08.785881 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:08.786580 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:08.786585 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:08.786588 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:08.786737 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:08.787112 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:08.787117 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:08.787119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:08.787466 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:08.787600 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:08.788271 (+ 671us) server_negotiation.cc:299] Negotiation successful
0808 23:30:08.788486 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 23:30:08.789479 32335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:08.772713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.66:34581)
0808 23:30:08.773386 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:08.773429 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:08.775002 (+ 1573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:08.775733 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:08.775742 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:08.776214 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:30:08.776738 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:08.776755 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:08.777735 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:08.777739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:08.778928 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:08.778940 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:08.779880 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:08.779889 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:08.780093 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:08.780693 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:30:08.780714 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:30:08.782758 (+ 2044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:30:08.786010 (+ 3252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:08.786016 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:08.786027 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:08.786458 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:08.786833 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:08.786836 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:08.786839 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:08.787016 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:08.787584 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:30:08.787589 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:30:08.787930 (+ 341us) client_negotiation.cc:769] Sending connection context
0808 23:30:08.788146 (+ 216us) client_negotiation.cc:240] Negotiation successful
0808 23:30:08.788492 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":265,"threads_started":1}
I20260808 23:30:08.790843 32333 heartbeater.cc:347] Connected to a master server at 127.9.44.126:37873
I20260808 23:30:08.791138 32333 heartbeater.cc:464] Registering TS with master...
I20260808 23:30:08.791905 32333 heartbeater.cc:511] Master 127.9.44.126:37873 requested a full tablet report, sending...
I20260808 23:30:08.793344 31899 ts_manager.cc:194] Registered new tserver with Master: 06d7d5fb6584491faf26ab94a4341c39 (127.9.44.66:33305)
I20260808 23:30:08.794392 31899 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.66@KRBTEST.COM'} at 127.9.44.66:34581
W20260808 23:30:09.106676 32339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:30:09.106936 32339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:30:09.107038 32339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:30:09.114048 32339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:30:09.114141 32339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:30:09.114255 32339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:30:09.123528 32339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:43633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:44281
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=36225
--tserver_master_addrs=127.9.44.126:37873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:30:09.124666 32339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:30:09.125962 32339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:30:09.138134 32348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:09.138250 32347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:09.138237 32350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:30:09.139094 32339 server_base.cc:1034] running on GCE node
I20260808 23:30:09.139623 32339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:30:09.140512 32339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:30:09.141708 32339 hybrid_clock.cc:648] HybridClock initialized: now 1786231809141668 us; error 26 us; skew 500 ppm
I20260808 23:30:09.142027 32339 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:30:09 dist-test-slave-6bbx krb5kdc[31068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231809, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:30:09.145294 32339 init.cc:377] Logged in from keytab as kudu/127.9.44.67@KRBTEST.COM (short username kudu)
I20260808 23:30:09.146581 32339 webserver.cc:460] Webserver started at http://127.9.44.67:36225/ using document root <none> and password file <none>
I20260808 23:30:09.147218 32339 fs_manager.cc:359] Metadata directory not provided
I20260808 23:30:09.147346 32339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:30:09.152330 32339 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:30:09.154820 32360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:09.156100 32339 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:30:09.156252 32339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec"
format_stamp: "Formatted at 2026-08-08 23:30:06 on dist-test-slave-6bbx"
server_key: "d5f6ccb0e5f9c32337e15084c110d63e"
server_key_iv: "338032937239cb6996982512e618e913"
server_key_version: "encryptionkey@0"
I20260808 23:30:09.157178 32339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:30:09.178725 32339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:30:09.182237 32339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:30:09.182464 32339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:30:09.183334 32339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:30:09.184607 32339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:30:09.184707 32339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:09.184806 32339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:30:09.184876 32339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:09.214600 32339 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:44281
I20260808 23:30:09.214663 32514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:44281 every 8 connection(s)
I20260808 23:30:09.215967 32339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:30:09.217526 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 32339
Aug 08 23:30:09 dist-test-slave-6bbx krb5kdc[31068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231809, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:30:09.239470 32148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:09.218509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49874 (local address 127.9.44.126:37873)
0808 23:30:09.218740 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:09.218746 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:09.224423 (+ 5677us) server_negotiation.cc:407] Connection header received
0808 23:30:09.224593 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:09.224598 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:09.224659 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:09.224758 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:09.226245 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.227313 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.229198 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.229735 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.229857 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:09.233049 (+ 3192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:09.233056 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:09.233060 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:09.233096 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:09.236564 (+ 3468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:09.237408 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:09.237429 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:09.237433 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:09.237590 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:09.238021 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:09.238027 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:09.238030 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:09.238384 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:09.238606 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:09.238988 (+ 382us) server_negotiation.cc:299] Negotiation successful
0808 23:30:09.239166 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":66}
I20260808 23:30:09.240953 32520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:09.218627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:47069 (local address 127.9.44.126:37873)
0808 23:30:09.224012 (+ 5385us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:09.224016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:09.224031 (+ 15us) server_negotiation.cc:407] Connection header received
0808 23:30:09.224081 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:09.224085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:09.224142 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:09.224255 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:09.226093 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.227313 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.229498 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.230115 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.230198 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:09.233944 (+ 3746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:09.233950 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:09.233955 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:09.234003 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:09.237503 (+ 3500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:09.238441 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:09.238448 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:09.238451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:09.238599 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:09.239080 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:09.239085 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:09.239089 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:09.239422 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:09.239545 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:09.240392 (+ 847us) server_negotiation.cc:299] Negotiation successful
0808 23:30:09.240590 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":5290,"thread_start_us":5193,"threads_started":1}
I20260808 23:30:09.241880 32522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:09.219070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.67:47069)
0808 23:30:09.219879 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:09.219923 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:09.221588 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:09.224585 (+ 2997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:09.224596 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:09.225253 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:30:09.225920 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:09.225940 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.227489 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.227494 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:09.229366 (+ 1872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:09.229379 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.230342 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.230352 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:09.230586 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:09.231156 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:30:09.231185 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:30:09.233730 (+ 2545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:30:09.237676 (+ 3946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:09.237685 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:09.237699 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:09.238312 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:09.238722 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:09.238725 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:09.238728 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:09.238944 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:09.239556 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:30:09.239563 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:30:09.239986 (+ 423us) client_negotiation.cc:769] Sending connection context
0808 23:30:09.240268 (+ 282us) client_negotiation.cc:240] Negotiation successful
0808 23:30:09.240623 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":325,"threads_started":1}
I20260808 23:30:09.242992 32515 heartbeater.cc:347] Connected to a master server at 127.9.44.126:37873
I20260808 23:30:09.243270 32515 heartbeater.cc:464] Registering TS with master...
I20260808 23:30:09.243978 32515 heartbeater.cc:511] Master 127.9.44.126:37873 requested a full tablet report, sending...
I20260808 23:30:09.245268 31899 ts_manager.cc:194] Registered new tserver with Master: ca66f7e2cb6e47a1a4820f72a41e40ec (127.9.44.67:44281)
I20260808 23:30:09.245994 31899 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.67@KRBTEST.COM'} at 127.9.44.67:47069
I20260808 23:30:09.254139 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:30:09.260418 32520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:09.254996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49884 (local address 127.9.44.126:37873)
0808 23:30:09.255151 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:09.255155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:09.255572 (+ 417us) server_negotiation.cc:407] Connection header received
0808 23:30:09.255800 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:09.255804 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:09.255932 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:09.256037 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:30:09.256440 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.257139 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.258594 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.259008 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.259093 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:09.259521 (+ 428us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:30:09.259982 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:30:09.260084 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:09.260185 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 23:30:09.260244 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 23:30:09.261224 31899 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:49884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:30:09.268405 31899 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:30:09.290711 32529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:09.279591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57990 (local address 127.9.44.65:44937)
0808 23:30:09.280398 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:09.280408 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:09.281019 (+ 611us) server_negotiation.cc:407] Connection header received
0808 23:30:09.281445 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:09.281454 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:09.281697 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:09.281871 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:30:09.282904 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.284440 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.287450 (+ 3010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.289141 (+ 1691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.289251 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:09.289445 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:09.290159 (+ 714us) server_negotiation.cc:299] Negotiation successful
0808 23:30:09.290344 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":463,"threads_started":1}
I20260808 23:30:09.291759 32528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:09.279425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:44937 (local address 127.0.0.1:57990)
0808 23:30:09.280858 (+ 1433us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:09.280907 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:09.281195 (+ 288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:09.282209 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:09.282217 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:09.282249 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:30:09.282684 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:09.282697 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.284636 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.284641 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:09.287266 (+ 2625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:09.287284 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.289683 (+ 2399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.289691 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:09.289894 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:09.289938 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 23:30:09.291165 (+ 1227us) client_negotiation.cc:240] Negotiation successful
0808 23:30:09.291351 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1186,"thread_start_us":1011,"threads_started":1}
I20260808 23:30:09.296233 32058 tablet_service.cc:1467] Processing CreateTablet for tablet e238657407494aa0b315cc71d08130e6 (DEFAULT_TABLE table=test-table [id=3ba038a414d34d078fe6713eedb6cdcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:30:09.297634 32058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e238657407494aa0b315cc71d08130e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:30:09.299333 32528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:09.281239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:33305 (local address 127.0.0.1:47122)
0808 23:30:09.292130 (+ 10891us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:09.292147 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:09.292245 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:09.293850 (+ 1605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:09.293854 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:09.293878 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:30:09.294019 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:09.294030 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.295241 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.295246 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:09.296810 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:09.296826 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.298607 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.298614 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:09.298818 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:09.298850 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 23:30:09.298942 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 23:30:09.299077 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10784,"thread_start_us":11606,"threads_started":1}
I20260808 23:30:09.299453 32537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:09.281873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47122 (local address 127.9.44.66:33305)
0808 23:30:09.293350 (+ 11477us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:09.293357 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:09.293389 (+ 32us) server_negotiation.cc:407] Connection header received
0808 23:30:09.293485 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:09.293492 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:09.293683 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:09.293839 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:30:09.294160 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.295097 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.296999 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.298416 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.298555 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:09.298749 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:09.299008 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 23:30:09.299171 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11318,"thread_start_us":11212,"threads_started":1}
I20260808 23:30:09.302058 32530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:09.280174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:44281 (local address 127.0.0.1:56466)
0808 23:30:09.281113 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:09.281129 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:09.281247 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:09.293371 (+ 12124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:09.293375 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:09.293401 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:30:09.293700 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:09.293713 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.297716 (+ 4003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.297721 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:09.299603 (+ 1882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:09.299617 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.301335 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.301341 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:09.301518 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:09.301552 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 23:30:09.301645 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 23:30:09.301764 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":820,"thread_start_us":843,"threads_started":1}
I20260808 23:30:09.302839 32533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:09.280755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56466 (local address 127.9.44.67:44281)
0808 23:30:09.292810 (+ 12055us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:09.292817 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:09.292850 (+ 33us) server_negotiation.cc:407] Connection header received
0808 23:30:09.292941 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:09.292948 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:09.293172 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:09.293348 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:30:09.293926 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.297548 (+ 3622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.299768 (+ 2220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.301117 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.301242 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:09.301446 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:09.301720 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 23:30:09.302497 (+ 777us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11886,"thread_start_us":11729,"threads_started":1}
I20260808 23:30:09.302836 32248 tablet_service.cc:1467] Processing CreateTablet for tablet e238657407494aa0b315cc71d08130e6 (DEFAULT_TABLE table=test-table [id=3ba038a414d34d078fe6713eedb6cdcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:30:09.304111 32248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e238657407494aa0b315cc71d08130e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:30:09.306385 32420 tablet_service.cc:1467] Processing CreateTablet for tablet e238657407494aa0b315cc71d08130e6 (DEFAULT_TABLE table=test-table [id=3ba038a414d34d078fe6713eedb6cdcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:30:09.307724 32420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e238657407494aa0b315cc71d08130e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:30:09.310698 32540 tablet_bootstrap.cc:492] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7: Bootstrap starting.
I20260808 23:30:09.313722 32540 tablet_bootstrap.cc:654] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7: Neither blocks nor log segments found. Creating new log.
I20260808 23:30:09.314674 32540 log.cc:824] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7: Log is configured to *not* fsync() on all Append() calls
I20260808 23:30:09.317147 32540 tablet_bootstrap.cc:492] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7: No bootstrap required, opened a new log
I20260808 23:30:09.317459 32540 ts_tablet_manager.cc:1397] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 23:30:09.319021 32543 tablet_bootstrap.cc:492] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39: Bootstrap starting.
I20260808 23:30:09.321911 32544 tablet_bootstrap.cc:492] T e238657407494aa0b315cc71d08130e6 P ca66f7e2cb6e47a1a4820f72a41e40ec: Bootstrap starting.
I20260808 23:30:09.322433 32543 tablet_bootstrap.cc:654] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39: Neither blocks nor log segments found. Creating new log.
I20260808 23:30:09.323697 32543 log.cc:824] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39: Log is configured to *not* fsync() on all Append() calls
I20260808 23:30:09.324911 32544 tablet_bootstrap.cc:654] T e238657407494aa0b315cc71d08130e6 P ca66f7e2cb6e47a1a4820f72a41e40ec: Neither blocks nor log segments found. Creating new log.
I20260808 23:30:09.326021 32544 log.cc:824] T e238657407494aa0b315cc71d08130e6 P ca66f7e2cb6e47a1a4820f72a41e40ec: Log is configured to *not* fsync() on all Append() calls
I20260808 23:30:09.326786 32543 tablet_bootstrap.cc:492] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39: No bootstrap required, opened a new log
I20260808 23:30:09.327040 32543 ts_tablet_manager.cc:1397] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260808 23:30:09.327270 32540 raft_consensus.cc:348] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } } peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } }
I20260808 23:30:09.327721 32540 raft_consensus.cc:374] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:30:09.327867 32540 raft_consensus.cc:729] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d818e1e5401472f9f3b0a667fee06b7, State: Initialized, Role: FOLLOWER
I20260808 23:30:09.328661 32544 tablet_bootstrap.cc:492] T e238657407494aa0b315cc71d08130e6 P ca66f7e2cb6e47a1a4820f72a41e40ec: No bootstrap required, opened a new log
I20260808 23:30:09.328558 32540 consensus_queue.cc:260] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } } peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } }
I20260808 23:30:09.328966 32544 ts_tablet_manager.cc:1397] T e238657407494aa0b315cc71d08130e6 P ca66f7e2cb6e47a1a4820f72a41e40ec: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 23:30:09.329787 32146 heartbeater.cc:503] Master 127.9.44.126:37873 was elected leader, sending a full tablet report...
I20260808 23:30:09.330369 32540 ts_tablet_manager.cc:1428] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7: Time spent starting tablet: real 0.013s user 0.008s sys 0.005s
I20260808 23:30:09.335548 32543 raft_consensus.cc:348] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } } peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } }
I20260808 23:30:09.335932 32543 raft_consensus.cc:374] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:30:09.336023 32543 raft_consensus.cc:729] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06d7d5fb6584491faf26ab94a4341c39, State: Initialized, Role: FOLLOWER
I20260808 23:30:09.336548 32543 consensus_queue.cc:260] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } } peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } }
I20260808 23:30:09.337581 32333 heartbeater.cc:503] Master 127.9.44.126:37873 was elected leader, sending a full tablet report...
I20260808 23:30:09.338282 32543 ts_tablet_manager.cc:1428] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39: Time spent starting tablet: real 0.011s user 0.011s sys 0.001s
I20260808 23:30:09.338771 32544 raft_consensus.cc:348] T e238657407494aa0b315cc71d08130e6 P ca66f7e2cb6e47a1a4820f72a41e40ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } } peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } }
I20260808 23:30:09.339138 32544 raft_consensus.cc:374] T e238657407494aa0b315cc71d08130e6 P ca66f7e2cb6e47a1a4820f72a41e40ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:30:09.339231 32544 raft_consensus.cc:729] T e238657407494aa0b315cc71d08130e6 P ca66f7e2cb6e47a1a4820f72a41e40ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca66f7e2cb6e47a1a4820f72a41e40ec, State: Initialized, Role: FOLLOWER
I20260808 23:30:09.339793 32544 consensus_queue.cc:260] T e238657407494aa0b315cc71d08130e6 P ca66f7e2cb6e47a1a4820f72a41e40ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } } peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } }
W20260808 23:30:09.340229 32147 tablet.cc:2369] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:30:09.344230 32515 heartbeater.cc:503] Master 127.9.44.126:37873 was elected leader, sending a full tablet report...
I20260808 23:30:09.344817 32544 ts_tablet_manager.cc:1428] T e238657407494aa0b315cc71d08130e6 P ca66f7e2cb6e47a1a4820f72a41e40ec: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
I20260808 23:30:09.367795 32550 raft_consensus.cc:482] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:30:09.368017 32550 raft_consensus.cc:504] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } } peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } }
I20260808 23:30:09.369470 32550 leader_election.cc:288] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d818e1e5401472f9f3b0a667fee06b7 (127.9.44.65:44937), ca66f7e2cb6e47a1a4820f72a41e40ec (127.9.44.67:44281)
I20260808 23:30:09.375339 32529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:09.369781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:32807 (local address 127.9.44.65:44937)
0808 23:30:09.370034 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:09.370041 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:09.370394 (+ 353us) server_negotiation.cc:407] Connection header received
0808 23:30:09.370572 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:09.370575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:09.370628 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:09.370699 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:30:09.371202 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.371964 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.373727 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.374474 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.374592 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:09.374679 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:09.374980 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 23:30:09.375100 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260808 23:30:09.375481 32552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:09.369665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:44937 (local address 127.9.44.66:32807)
0808 23:30:09.370296 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:09.370312 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:09.370452 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:09.370741 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:09.370745 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:09.370771 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:30:09.371071 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:09.371080 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.372083 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.372088 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:09.373584 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:09.373595 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.374680 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.374686 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:09.374825 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:09.374848 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 23:30:09.374931 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 23:30:09.375085 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":38,"thread_start_us":327,"threads_started":1}
I20260808 23:30:09.375684 32533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:09.369922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:54675 (local address 127.9.44.67:44281)
0808 23:30:09.370080 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:09.370086 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:09.370674 (+ 588us) server_negotiation.cc:407] Connection header received
0808 23:30:09.370827 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:09.370831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:09.370893 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:09.370965 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:30:09.371529 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.372244 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.373946 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.374722 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.374835 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:09.374942 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:09.375313 (+ 371us) server_negotiation.cc:299] Negotiation successful
0808 23:30:09.375434 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 23:30:09.375952 32555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:09.369714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:44281 (local address 127.9.44.66:54675)
0808 23:30:09.370571 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:09.370585 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:09.370706 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:09.371006 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:09.371011 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:09.371041 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:30:09.371373 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:09.371382 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.372392 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.372396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:09.373793 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:09.373806 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.374930 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.374936 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:09.375146 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:09.375171 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 23:30:09.375254 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 23:30:09.375382 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"mutex_wait_us":45,"thread_start_us":569,"threads_started":1}
I20260808 23:30:09.376861 32085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e238657407494aa0b315cc71d08130e6" candidate_uuid: "06d7d5fb6584491faf26ab94a4341c39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d818e1e5401472f9f3b0a667fee06b7" is_pre_election: true
I20260808 23:30:09.377025 32445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e238657407494aa0b315cc71d08130e6" candidate_uuid: "06d7d5fb6584491faf26ab94a4341c39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" is_pre_election: true
I20260808 23:30:09.377444 32445 raft_consensus.cc:2454] T e238657407494aa0b315cc71d08130e6 P ca66f7e2cb6e47a1a4820f72a41e40ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d7d5fb6584491faf26ab94a4341c39 in term 0.
I20260808 23:30:09.377444 32085 raft_consensus.cc:2454] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06d7d5fb6584491faf26ab94a4341c39 in term 0.
I20260808 23:30:09.378105 32174 leader_election.cc:302] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d7d5fb6584491faf26ab94a4341c39, ca66f7e2cb6e47a1a4820f72a41e40ec; no voters:
I20260808 23:30:09.378446 32550 raft_consensus.cc:2781] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:30:09.378600 32550 raft_consensus.cc:482] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:30:09.378690 32550 raft_consensus.cc:3037] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:30:09.380775 32550 raft_consensus.cc:504] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } } peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } }
I20260808 23:30:09.381287 32550 leader_election.cc:288] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39 [CANDIDATE]: Term 1 election: Requested vote from peers 7d818e1e5401472f9f3b0a667fee06b7 (127.9.44.65:44937), ca66f7e2cb6e47a1a4820f72a41e40ec (127.9.44.67:44281)
I20260808 23:30:09.381661 32085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e238657407494aa0b315cc71d08130e6" candidate_uuid: "06d7d5fb6584491faf26ab94a4341c39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d818e1e5401472f9f3b0a667fee06b7"
I20260808 23:30:09.381743 32445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e238657407494aa0b315cc71d08130e6" candidate_uuid: "06d7d5fb6584491faf26ab94a4341c39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec"
I20260808 23:30:09.381819 32085 raft_consensus.cc:3037] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:30:09.381919 32445 raft_consensus.cc:3037] T e238657407494aa0b315cc71d08130e6 P ca66f7e2cb6e47a1a4820f72a41e40ec [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:30:09.383962 32085 raft_consensus.cc:2454] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d7d5fb6584491faf26ab94a4341c39 in term 1.
I20260808 23:30:09.384130 32445 raft_consensus.cc:2454] T e238657407494aa0b315cc71d08130e6 P ca66f7e2cb6e47a1a4820f72a41e40ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06d7d5fb6584491faf26ab94a4341c39 in term 1.
I20260808 23:30:09.384338 32172 leader_election.cc:302] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06d7d5fb6584491faf26ab94a4341c39, 7d818e1e5401472f9f3b0a667fee06b7; no voters:
I20260808 23:30:09.384620 32550 raft_consensus.cc:2781] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:30:09.384999 32550 raft_consensus.cc:686] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39 [term 1 LEADER]: Becoming Leader. State: Replica: 06d7d5fb6584491faf26ab94a4341c39, State: Running, Role: LEADER
I20260808 23:30:09.385440 32550 consensus_queue.cc:237] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } } peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } }
I20260808 23:30:09.389578 31899 catalog_manager.cc:5495] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06d7d5fb6584491faf26ab94a4341c39 (127.9.44.66). New cstate: current_term: 1 leader_uuid: "06d7d5fb6584491faf26ab94a4341c39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260808 23:30:09.410738 32537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:09.405704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47132 (local address 127.9.44.66:33305)
0808 23:30:09.405914 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:09.405920 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:09.405938 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:30:09.406077 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:09.406082 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:09.406160 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:09.406279 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:30:09.406646 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.407323 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.408726 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.409136 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.409245 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:09.409718 (+ 473us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:30:09.410158 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:30:09.410289 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:09.410409 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 23:30:09.410485 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 23:30:09.418185 32085 raft_consensus.cc:1261] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 1 FOLLOWER]: Refusing update from remote peer 06d7d5fb6584491faf26ab94a4341c39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:30:09.418193 32445 raft_consensus.cc:1261] T e238657407494aa0b315cc71d08130e6 P ca66f7e2cb6e47a1a4820f72a41e40ec [term 1 FOLLOWER]: Refusing update from remote peer 06d7d5fb6584491faf26ab94a4341c39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:30:09.419032 32557 consensus_queue.cc:1035] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:30:09.419348 32550 consensus_queue.cc:1035] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:30:09.432004 32562 mvcc.cc:205] Tried to move back new op lower bound from 7316405491367882752 to 7316405491246432256. Current Snapshot: MvccSnapshot[applied={T|T < 7316405491367882752}]
I20260808 23:30:09.443722 31897 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:49884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:30:09.444037 31897 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:49884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:30:09.447576 31897 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2: Sending DeleteTablet for 3 replicas of tablet e238657407494aa0b315cc71d08130e6
I20260808 23:30:09.448647 32058 tablet_service.cc:1514] Processing DeleteTablet for tablet e238657407494aa0b315cc71d08130e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:30:09 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:57990
I20260808 23:30:09.448731 32248 tablet_service.cc:1514] Processing DeleteTablet for tablet e238657407494aa0b315cc71d08130e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:30:09 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:47122
I20260808 23:30:09.448808 32420 tablet_service.cc:1514] Processing DeleteTablet for tablet e238657407494aa0b315cc71d08130e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:30:09 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:56466
I20260808 23:30:09.449990 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 31953
I20260808 23:30:09.468706 32529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:09.450704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58002 (local address 127.9.44.65:44937)
0808 23:30:09.450874 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:09.450881 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:09.451028 (+ 147us) server_negotiation.cc:407] Connection header received
0808 23:30:09.451203 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:09.451217 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:09.451293 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:09.451394 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:09.453246 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.454447 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.456457 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.457147 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.457296 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:09.461751 (+ 4455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:09.461765 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:09.461783 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:09.461826 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:09.465633 (+ 3807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:09.466437 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:09.466446 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:09.466450 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:09.466610 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:09.466994 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:09.466999 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:09.467002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:09.467472 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:09.467760 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:09.468094 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 23:30:09.468324 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 23:30:09.469336 32140 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:30:09.470762 32577 tablet_replica.cc:330] T e238657407494aa0b315cc71d08130e6 P ca66f7e2cb6e47a1a4820f72a41e40ec: stopping tablet replica
I20260808 23:30:09.471184 32577 raft_consensus.cc:2229] T e238657407494aa0b315cc71d08130e6 P ca66f7e2cb6e47a1a4820f72a41e40ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:30:09.471740 32577 raft_consensus.cc:2258] T e238657407494aa0b315cc71d08130e6 P ca66f7e2cb6e47a1a4820f72a41e40ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:30:09.475559 32577 ts_tablet_manager.cc:1905] T e238657407494aa0b315cc71d08130e6 P ca66f7e2cb6e47a1a4820f72a41e40ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:30:09.477831 32575 tablet_replica.cc:330] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39: stopping tablet replica
I20260808 23:30:09.478940 32575 raft_consensus.cc:2229] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:30:09.479858 32575 raft_consensus.cc:2258] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:30:09.481878 32575 ts_tablet_manager.cc:1905] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:30:09.484730 32577 ts_tablet_manager.cc:1918] T e238657407494aa0b315cc71d08130e6 P ca66f7e2cb6e47a1a4820f72a41e40ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:30:09.484946 32577 log.cc:1196] T e238657407494aa0b315cc71d08130e6 P ca66f7e2cb6e47a1a4820f72a41e40ec: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/e238657407494aa0b315cc71d08130e6
I20260808 23:30:09.485541 32577 ts_tablet_manager.cc:1939] T e238657407494aa0b315cc71d08130e6 P ca66f7e2cb6e47a1a4820f72a41e40ec: Deleting consensus metadata
I20260808 23:30:09.487432 31877 catalog_manager.cc:4841] TS ca66f7e2cb6e47a1a4820f72a41e40ec (127.9.44.67:44281): tablet e238657407494aa0b315cc71d08130e6 (table test-table [id=3ba038a414d34d078fe6713eedb6cdcc]) successfully deleted
I20260808 23:30:09.489317 32575 ts_tablet_manager.cc:1918] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:30:09.489486 32575 log.cc:1196] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/e238657407494aa0b315cc71d08130e6
I20260808 23:30:09.490022 32575 ts_tablet_manager.cc:1939] T e238657407494aa0b315cc71d08130e6 P 06d7d5fb6584491faf26ab94a4341c39: Deleting consensus metadata
I20260808 23:30:09.491701 31876 catalog_manager.cc:4841] TS 06d7d5fb6584491faf26ab94a4341c39 (127.9.44.66:33305): tablet e238657407494aa0b315cc71d08130e6 (table test-table [id=3ba038a414d34d078fe6713eedb6cdcc]) successfully deleted
I20260808 23:30:09.568650 32576 tablet_replica.cc:330] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7: stopping tablet replica
I20260808 23:30:09.568857 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 31953
W20260808 23:30:09.591595 31876 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.44.65:44937 (error 108)
I20260808 23:30:09.592135 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 32150
I20260808 23:30:09.593674 32530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:09.593084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:44937 (local address 127.0.0.1:58018)
0808 23:30:09.593357 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:09.593491 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260808 23:30:09.593935 31876 catalog_manager.cc:4584] TS 7d818e1e5401472f9f3b0a667fee06b7 (127.9.44.65:44937): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e238657407494aa0b315cc71d08130e6: Network error: Client connection negotiation failed: client connection to 127.9.44.65:44937: connect: Connection refused (error 111)
I20260808 23:30:09.609562 32537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:09.593978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47142 (local address 127.9.44.66:33305)
0808 23:30:09.594177 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:09.594183 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:09.594201 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:30:09.594273 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:09.594279 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:09.594349 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:09.594431 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:09.596209 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.597011 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.598911 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.599339 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.599425 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:09.602635 (+ 3210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:09.602647 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:09.602664 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:09.602695 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:09.605546 (+ 2851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:09.606376 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:09.606386 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:09.606389 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:09.606569 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:09.607031 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:09.607037 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:09.607040 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:09.607609 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:09.607957 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:09.608618 (+ 661us) server_negotiation.cc:299] Negotiation successful
0808 23:30:09.609041 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 23:30:09.610234 32326 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:30:09.647572 32530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:09.647073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:44937 (local address 127.0.0.1:58030)
0808 23:30:09.647282 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:09.647392 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 23:30:09.648689 32530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:09.648325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:44937 (local address 127.0.0.1:58034)
0808 23:30:09.648462 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:09.648548 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 23:30:09.709225 32530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:09.708610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:44937 (local address 127.0.0.1:58048)
0808 23:30:09.708899 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:09.709013 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260808 23:30:09.710270 32530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:09.709894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:44937 (local address 127.0.0.1:58054)
0808 23:30:09.710041 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:09.710128 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 23:30:09.721031 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 32150
I20260808 23:30:09.744057 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 32339
I20260808 23:30:09.759413 32533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:09.744825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56472 (local address 127.9.44.67:44281)
0808 23:30:09.744995 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:09.745001 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:09.745066 (+ 65us) server_negotiation.cc:407] Connection header received
0808 23:30:09.745256 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:09.745260 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:09.745326 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:09.745423 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:09.746778 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.747554 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.748819 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:09.749221 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:09.749334 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:09.754505 (+ 5171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:09.754519 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:09.754535 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:09.754572 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:09.757011 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:09.757685 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:09.757692 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:09.757694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:09.757848 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:09.758144 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:09.758148 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:09.758151 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:09.758521 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:09.758738 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:09.758991 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 23:30:09.759159 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 23:30:09.760046 32507 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:30:09.790979 32530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:09.790547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:44937 (local address 127.0.0.1:58060)
0808 23:30:09.790738 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:09.790823 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 23:30:09.792009 32530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:09.791602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:44937 (local address 127.0.0.1:58064)
0808 23:30:09.791827 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:09.791900 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260808 23:30:09.847740 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 32339
I20260808 23:30:09.871524 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 31853
I20260808 23:30:09.872812 31942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:30:09.981539 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 31853
I20260808 23:30:09.981585 32530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:09.980815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:44937 (local address 127.0.0.1:58070)
0808 23:30:09.981026 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:09.981171 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:44937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 23:30:10.005580 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:37873
--webserver_interface=127.9.44.126
--webserver_port=39719
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:43633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:37873
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:30:10.328011 32585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:30:10.328285 32585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:30:10.328425 32585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:30:10.335222 32585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:30:10.335327 32585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:30:10.335403 32585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:30:10.335482 32585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:30:10.335543 32585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:30:10.344656 32585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:43633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:37873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:37873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=39719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:30:10.345862 32585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:30:10.347056 32585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:30:10.358001 32592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:10.358038 32595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:10.357949 32593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:30:10.358769 32585 server_base.cc:1034] running on GCE node
I20260808 23:30:10.359469 32585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:30:10.360742 32585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:30:10.361939 32585 hybrid_clock.cc:648] HybridClock initialized: now 1786231810361883 us; error 47 us; skew 500 ppm
I20260808 23:30:10.362274 32585 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:30:10 dist-test-slave-6bbx krb5kdc[31068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231810, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:30:10.365201 32585 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:30:10.366503 32585 webserver.cc:460] Webserver started at http://127.9.44.126:39719/ using document root <none> and password file <none>
I20260808 23:30:10.367111 32585 fs_manager.cc:359] Metadata directory not provided
I20260808 23:30:10.367234 32585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:30:10.372267 32585 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 23:30:10.374874 32603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:10.376082 32585 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 23:30:10.376241 32585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "150ef7b807d049cf8484b15fdb4b98a2"
format_stamp: "Formatted at 2026-08-08 23:30:04 on dist-test-slave-6bbx"
server_key: "00212782afc4ba3e063c71c271b67ed7"
server_key_iv: "5cfb7381cd0f45dfd5ea7ccfe9149ed4"
server_key_version: "encryptionkey@0"
I20260808 23:30:10.377107 32585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:30:10.390053 32585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:30:10.393867 32585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:30:10.394093 32585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:30:10.410719 32585 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:37873
I20260808 23:30:10.410753 32669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:37873 every 8 connection(s)
I20260808 23:30:10.412045 32585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:30:10.418584 32671 sys_catalog.cc:263] Verifying existing consensus state
I20260808 23:30:10.419766 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 32585
I20260808 23:30:10.420526 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:44937
--local_ip_for_outbound_sockets=127.9.44.65
--tserver_master_addrs=127.9.44.126:37873
--webserver_port=39615
--webserver_interface=127.9.44.65
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:43633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:30:10.421775 32671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2: Bootstrap starting.
I20260808 23:30:10.444021 32671 log.cc:824] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2: Log is configured to *not* fsync() on all Append() calls
I20260808 23:30:10.465612 32671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:30:10.466212 32671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2: Bootstrap complete.
I20260808 23:30:10.479063 32671 raft_consensus.cc:348] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150ef7b807d049cf8484b15fdb4b98a2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37873 } }
I20260808 23:30:10.480341 32671 raft_consensus.cc:729] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 150ef7b807d049cf8484b15fdb4b98a2, State: Initialized, Role: FOLLOWER
I20260808 23:30:10.481058 32671 consensus_queue.cc:260] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150ef7b807d049cf8484b15fdb4b98a2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37873 } }
I20260808 23:30:10.481341 32671 raft_consensus.cc:388] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:30:10.481464 32671 raft_consensus.cc:482] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:30:10.481747 32671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [term 2 FOLLOWER]: Advancing to term 3
I20260808 23:30:10.487423 32671 raft_consensus.cc:504] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150ef7b807d049cf8484b15fdb4b98a2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37873 } }
I20260808 23:30:10.488039 32671 leader_election.cc:302] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 150ef7b807d049cf8484b15fdb4b98a2; no voters:
I20260808 23:30:10.488679 32671 leader_election.cc:288] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 23:30:10.488806 32676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 23:30:10.489900 32676 raft_consensus.cc:686] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [term 3 LEADER]: Becoming Leader. State: Replica: 150ef7b807d049cf8484b15fdb4b98a2, State: Running, Role: LEADER
I20260808 23:30:10.490442 32676 consensus_queue.cc:237] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150ef7b807d049cf8484b15fdb4b98a2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37873 } }
I20260808 23:30:10.490773 32671 sys_catalog.cc:564] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:30:10.492616 32678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 150ef7b807d049cf8484b15fdb4b98a2. Latest consensus state: current_term: 3 leader_uuid: "150ef7b807d049cf8484b15fdb4b98a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150ef7b807d049cf8484b15fdb4b98a2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37873 } } }
I20260808 23:30:10.492983 32678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [sys.catalog]: This master's current role is: LEADER
I20260808 23:30:10.493530 32677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "150ef7b807d049cf8484b15fdb4b98a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150ef7b807d049cf8484b15fdb4b98a2" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37873 } } }
I20260808 23:30:10.493775 32677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 150ef7b807d049cf8484b15fdb4b98a2 [sys.catalog]: This master's current role is: LEADER
I20260808 23:30:10.495954 32683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:30:10.506246 32683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:30:10.507791 32683 catalog_manager.cc:1259] Loaded cluster ID: 13cb099ddea348d39906a9fc7685aae3
I20260808 23:30:10.507948 32683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 23:30:10.510076 32683 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe534f8c9ad google::LogMessage::Fail() at ??:0
@ 0x7fe534f8f88a google::LogMessage::SendToLog() at ??:0
@ 0x7fe534f8c4de google::LogMessage::Flush() at ??:0
@ 0x7fe534f8da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe54a9bf512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fe54a9bc976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe54aa44610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fe54aa44439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fe537bd8345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe537bdb91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fe537bdb799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fe537bb6eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe53ae906db start_thread at ??:0
@ 0x7fe533e2661f clone at ??:0
W20260808 23:30:10.806748 32674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:30:10.806999 32674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:30:10.807114 32674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:30:10.813949 32674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:30:10.814056 32674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:30:10.814209 32674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:30:10.823428 32674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:43633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:44937
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=39615
--tserver_master_addrs=127.9.44.126:37873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:30:10.824563 32674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:30:10.825793 32674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:30:10.838230 32738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:10.838274 32741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:10.838186 32744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:30:10.838941 32674 server_base.cc:1034] running on GCE node
I20260808 23:30:10.839521 32674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:30:10.840426 32674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:30:10.841624 32674 hybrid_clock.cc:648] HybridClock initialized: now 1786231810841571 us; error 45 us; skew 500 ppm
I20260808 23:30:10.841913 32674 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:30:10 dist-test-slave-6bbx krb5kdc[31068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231810, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:30:10.844908 32674 init.cc:377] Logged in from keytab as kudu/127.9.44.65@KRBTEST.COM (short username kudu)
I20260808 23:30:10.846259 32674 webserver.cc:460] Webserver started at http://127.9.44.65:39615/ using document root <none> and password file <none>
I20260808 23:30:10.846890 32674 fs_manager.cc:359] Metadata directory not provided
I20260808 23:30:10.847019 32674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:30:10.852042 32674 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 23:30:10.857034 32753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:10.858415 32674 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 23:30:10.858605 32674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "7d818e1e5401472f9f3b0a667fee06b7"
format_stamp: "Formatted at 2026-08-08 23:30:05 on dist-test-slave-6bbx"
server_key: "773a831111f50dc2339c39bb037c4d3b"
server_key_iv: "9ffb05ece254f5ee1428a5acf459ea66"
server_key_version: "encryptionkey@0"
I20260808 23:30:10.859524 32674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:30:10.881479 32674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:30:10.885037 32674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:30:10.885260 32674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:30:10.886140 32674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:30:10.887847 32762 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260808 23:30:10.893230 32674 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 23:30:10.893348 32674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260808 23:30:10.893463 32674 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 23:30:10.896448 32674 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 23:30:10.896528 32674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20260808 23:30:10.896883 32762 tablet_bootstrap.cc:492] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7: Bootstrap starting.
I20260808 23:30:10.923532 32762 log.cc:824] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7: Log is configured to *not* fsync() on all Append() calls
I20260808 23:30:10.935222 32674 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:44937
I20260808 23:30:10.935279 429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:44937 every 8 connection(s)
I20260808 23:30:10.936182 32762 tablet_bootstrap.cc:492] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:30:10.936493 32674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:30:10.936694 32762 tablet_bootstrap.cc:492] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7: Bootstrap complete.
I20260808 23:30:10.938126 32762 ts_tablet_manager.cc:1397] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7: Time spent bootstrapping tablet: real 0.042s user 0.030s sys 0.008s
I20260808 23:30:10.941291 433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:10.939102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.65:58879)
0808 23:30:10.940028 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:10.940349 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":404,"threads_started":1}
W20260808 23:30:10.941859 431 heartbeater.cc:644] Failed to heartbeat to 127.9.44.126:37873 (0 consecutive failures): Network error: Failed to ping master at 127.9.44.126:37873: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
I20260808 23:30:10.942781 433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:10.942394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.65:56157)
0808 23:30:10.942535 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:10.942632 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 23:30:10.943188 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 32674
I20260808 23:30:10.943912 433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:10.943464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.65:42153)
0808 23:30:10.943630 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:10.943747 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 23:30:10.943940 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:33305
--local_ip_for_outbound_sockets=127.9.44.66
--tserver_master_addrs=127.9.44.126:37873
--webserver_port=46007
--webserver_interface=127.9.44.66
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:43633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:30:10.944137 431 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 23:30:10.945479 32762 raft_consensus.cc:348] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } } peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } }
I20260808 23:30:10.946312 32762 raft_consensus.cc:729] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d818e1e5401472f9f3b0a667fee06b7, State: Initialized, Role: FOLLOWER
I20260808 23:30:10.946874 32762 consensus_queue.cc:260] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } } peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } }
I20260808 23:30:10.948331 32762 ts_tablet_manager.cc:1428] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7: Time spent starting tablet: real 0.010s user 0.011s sys 0.001s
I20260808 23:30:10.948428 433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:10.948048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.65:42819)
0808 23:30:10.948207 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:10.948288 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 23:30:10.949229 433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:10.948952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.65:55003)
0808 23:30:10.949062 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:10.949121 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 23:30:11.265709 436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:30:11.265961 436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:30:11.266058 436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:30:11.272934 436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:30:11.273026 436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:30:11.273145 436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:30:11.282096 436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:43633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:33305
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=46007
--tserver_master_addrs=127.9.44.126:37873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:30:11.283177 436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:30:11.284410 436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:30:11.296940 445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:11.296871 449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:11.296978 446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:30:11.297641 436 server_base.cc:1034] running on GCE node
I20260808 23:30:11.298193 436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:30:11.299031 436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:30:11.300216 436 hybrid_clock.cc:648] HybridClock initialized: now 1786231811300165 us; error 45 us; skew 500 ppm
I20260808 23:30:11.300495 436 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:30:11 dist-test-slave-6bbx krb5kdc[31068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231811, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:30:11.303366 436 init.cc:377] Logged in from keytab as kudu/127.9.44.66@KRBTEST.COM (short username kudu)
I20260808 23:30:11.304708 436 webserver.cc:460] Webserver started at http://127.9.44.66:46007/ using document root <none> and password file <none>
I20260808 23:30:11.305346 436 fs_manager.cc:359] Metadata directory not provided
I20260808 23:30:11.305464 436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:30:11.310361 436 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 23:30:11.312865 457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:11.314036 436 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 23:30:11.314193 436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "06d7d5fb6584491faf26ab94a4341c39"
format_stamp: "Formatted at 2026-08-08 23:30:05 on dist-test-slave-6bbx"
server_key: "b08e2ace371eb613a9f211cb457242be"
server_key_iv: "91e68521b0fccdbd05ed51302fa941f1"
server_key_version: "encryptionkey@0"
I20260808 23:30:11.315091 436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:30:11.347499 436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:30:11.351074 436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:30:11.351321 436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:30:11.352293 436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:30:11.353523 436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:30:11.353621 436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:11.353731 436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:30:11.353803 436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:11.383419 436 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:33305
I20260808 23:30:11.383481 614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:33305 every 8 connection(s)
I20260808 23:30:11.384800 436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:30:11.389376 617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:11.387544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.66:41259)
0808 23:30:11.388318 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:11.388529 (+ 211us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":314,"threads_started":1}
I20260808 23:30:11.389638 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 436
W20260808 23:30:11.390058 615 heartbeater.cc:644] Failed to heartbeat to 127.9.44.126:37873 (0 consecutive failures): Network error: Failed to ping master at 127.9.44.126:37873: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
I20260808 23:30:11.390326 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:44281
--local_ip_for_outbound_sockets=127.9.44.67
--tserver_master_addrs=127.9.44.126:37873
--webserver_port=36225
--webserver_interface=127.9.44.67
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:43633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:30:11.390826 617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:11.390538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.66:49717)
0808 23:30:11.390659 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:11.390724 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 23:30:11.391577 617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:11.391298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.66:46841)
0808 23:30:11.391407 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:11.391475 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 23:30:11.391814 615 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 23:30:11.712415 618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:30:11.712656 618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:30:11.712785 618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:30:11.719818 618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:30:11.719913 618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:30:11.720057 618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:30:11.729154 618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:43633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:44281
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=36225
--tserver_master_addrs=127.9.44.126:37873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:30:11.730355 618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:30:11.731609 618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:30:11.744117 628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:11.744196 626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:11.744418 625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:30:11.745025 618 server_base.cc:1034] running on GCE node
I20260808 23:30:11.745556 618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:30:11.746387 618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:30:11.747563 618 hybrid_clock.cc:648] HybridClock initialized: now 1786231811747512 us; error 48 us; skew 500 ppm
I20260808 23:30:11.747875 618 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:30:11 dist-test-slave-6bbx krb5kdc[31068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231811, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:30:11.750712 618 init.cc:377] Logged in from keytab as kudu/127.9.44.67@KRBTEST.COM (short username kudu)
I20260808 23:30:11.752045 618 webserver.cc:460] Webserver started at http://127.9.44.67:36225/ using document root <none> and password file <none>
I20260808 23:30:11.752672 618 fs_manager.cc:359] Metadata directory not provided
I20260808 23:30:11.752789 618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:30:11.757671 618 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 23:30:11.760118 637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:11.761375 618 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 23:30:11.761516 618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec"
format_stamp: "Formatted at 2026-08-08 23:30:06 on dist-test-slave-6bbx"
server_key: "d5f6ccb0e5f9c32337e15084c110d63e"
server_key_iv: "338032937239cb6996982512e618e913"
server_key_version: "encryptionkey@0"
I20260808 23:30:11.762485 618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:30:11.797495 618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:30:11.800968 618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:30:11.801203 618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:30:11.802073 618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:30:11.803290 618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:30:11.803381 618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:11.803488 618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:30:11.803556 618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:11.832911 618 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:44281
I20260808 23:30:11.832975 787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:44281 every 8 connection(s)
I20260808 23:30:11.834399 618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:30:11.835599 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 618
I20260808 23:30:11.839228 791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:11.837523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.67:58945)
0808 23:30:11.838206 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:11.838415 (+ 209us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":259,"threads_started":1}
W20260808 23:30:11.839921 788 heartbeater.cc:644] Failed to heartbeat to 127.9.44.126:37873 (0 consecutive failures): Network error: Failed to ping master at 127.9.44.126:37873: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
I20260808 23:30:11.840647 791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:11.840352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.67:50141)
0808 23:30:11.840483 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:11.840547 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 23:30:11.841334 791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:11.841077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.67:58221)
0808 23:30:11.841172 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:11.841240 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 23:30:11.841527 788 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 23:30:11.950614 792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:11.949907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.65:38435)
0808 23:30:11.950338 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:11.950430 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":199,"threads_started":1}
I20260808 23:30:12.132248 793 raft_consensus.cc:482] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:30:12.132535 793 raft_consensus.cc:504] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } } peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } }
I20260808 23:30:12.134133 793 leader_election.cc:288] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06d7d5fb6584491faf26ab94a4341c39 (127.9.44.66:33305), ca66f7e2cb6e47a1a4820f72a41e40ec (127.9.44.67:44281)
Aug 08 23:30:12 dist-test-slave-6bbx krb5kdc[31068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231810, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for kudu/127.9.44.67@KRBTEST.COM
Aug 08 23:30:12 dist-test-slave-6bbx krb5kdc[31068](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231810, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for kudu/127.9.44.66@KRBTEST.COM
I20260808 23:30:12.159250 796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:12.134677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:33305 (local address 127.9.44.65:48823)
0808 23:30:12.136265 (+ 1588us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:12.136284 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:12.136399 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:12.139345 (+ 2946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:12.139368 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:12.140269 (+ 901us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:30:12.141010 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:12.141029 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:12.142549 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:12.142552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:12.143953 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:12.143966 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:12.144967 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:12.144976 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:12.145207 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:12.148967 (+ 3760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:30:12.148997 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:30:12.151585 (+ 2588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:30:12.155373 (+ 3788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:12.155377 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:12.155380 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:12.155987 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:12.156387 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:12.156391 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:12.156394 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:12.156647 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:12.157523 (+ 876us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:30:12.157530 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:30:12.157977 (+ 447us) client_negotiation.cc:769] Sending connection context
0808 23:30:12.158290 (+ 313us) client_negotiation.cc:240] Negotiation successful
0808 23:30:12.158735 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1459,"mutex_wait_us":261,"spinlock_wait_cycles":41472,"thread_start_us":1277,"threads_started":1}
I20260808 23:30:12.159332 794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:12.134593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:48823 (local address 127.9.44.66:33305)
0808 23:30:12.136699 (+ 2106us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:12.136725 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:12.136767 (+ 42us) server_negotiation.cc:407] Connection header received
0808 23:30:12.138459 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:12.138469 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:12.138841 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:12.139177 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:12.141193 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:12.142362 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:12.144087 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:12.144696 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:12.144876 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:12.151894 (+ 7018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:12.151904 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:12.151919 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:12.151962 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:12.155260 (+ 3298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:12.156102 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:12.156109 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:12.156116 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:12.156298 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:12.156756 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:12.156761 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:12.156765 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:12.157396 (+ 631us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:12.157591 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:12.158532 (+ 941us) server_negotiation.cc:299] Negotiation successful
0808 23:30:12.158920 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1913,"thread_start_us":1811,"threads_started":1}
I20260808 23:30:12.159794 792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:12.134592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:44281 (local address 127.9.44.65:58449)
0808 23:30:12.134851 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:12.134907 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:12.135510 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:12.139345 (+ 3835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:12.139356 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:12.140262 (+ 906us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:30:12.141009 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:12.141028 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:12.142542 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:12.142546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:12.143952 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:12.143965 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:12.144922 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:12.144929 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:12.145218 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:12.148916 (+ 3698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:30:12.148939 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:30:12.151267 (+ 2328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:30:12.155168 (+ 3901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:12.155175 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:12.155185 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:12.155907 (+ 722us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:12.156404 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:12.156408 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:12.156411 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:12.156647 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:12.157523 (+ 876us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:30:12.157530 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:30:12.157978 (+ 448us) client_negotiation.cc:769] Sending connection context
0808 23:30:12.158290 (+ 312us) client_negotiation.cc:240] Negotiation successful
0808 23:30:12.158860 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":391,"spinlock_wait_cycles":10368}
I20260808 23:30:12.159652 797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:12.134708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:58449 (local address 127.9.44.67:44281)
0808 23:30:12.136911 (+ 2203us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:12.136922 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:12.136957 (+ 35us) server_negotiation.cc:407] Connection header received
0808 23:30:12.138641 (+ 1684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:12.138650 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:12.138982 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:12.140046 (+ 1064us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:12.141198 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:12.142370 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:12.144084 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:12.144695 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:12.144878 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:12.151526 (+ 6648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:12.151533 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:12.151545 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:12.151582 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:12.155031 (+ 3449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:12.156042 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:12.156051 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:12.156059 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:12.156283 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:12.156756 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:12.156761 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:12.156765 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:12.157395 (+ 630us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:12.157574 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:12.158530 (+ 956us) server_negotiation.cc:299] Negotiation successful
0808 23:30:12.158920 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2052,"thread_start_us":1904,"threads_started":1}
W20260808 23:30:12.162801 32754 leader_election.cc:341] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 06d7d5fb6584491faf26ab94a4341c39 (127.9.44.66:33305): Not found: Tablet not found: e238657407494aa0b315cc71d08130e6
W20260808 23:30:12.163070 32756 leader_election.cc:341] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca66f7e2cb6e47a1a4820f72a41e40ec (127.9.44.67:44281): Not found: Tablet not found: e238657407494aa0b315cc71d08130e6
I20260808 23:30:12.163234 32756 leader_election.cc:302] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d818e1e5401472f9f3b0a667fee06b7; no voters: 06d7d5fb6584491faf26ab94a4341c39, ca66f7e2cb6e47a1a4820f72a41e40ec
I20260808 23:30:12.163625 793 raft_consensus.cc:2726] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 23:30:12.393028 799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:12.392259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.66:35175)
0808 23:30:12.392685 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:12.392806 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":197,"threads_started":1}
I20260808 23:30:12.842650 801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:12.841973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.67:49469)
0808 23:30:12.842362 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:12.842463 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":199,"threads_started":1}
I20260808 23:30:12.951984 802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:12.951354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.65:54223)
0808 23:30:12.951763 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:12.951840 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":204,"threads_started":1}
I20260808 23:30:13.394435 803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:13.393791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.66:33509)
0808 23:30:13.394191 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:13.394270 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":211,"threads_started":1}
I20260808 23:30:13.844128 805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:13.843409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.67:55163)
0808 23:30:13.843873 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:13.843948 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":245,"threads_started":1}
I20260808 23:30:13.953436 806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:13.952721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.65:47855)
0808 23:30:13.953183 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:13.953256 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":205,"threads_started":1}
I20260808 23:30:14.190254 807 raft_consensus.cc:482] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:30:14.190454 807 raft_consensus.cc:504] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } } peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } }
I20260808 23:30:14.191108 807 leader_election.cc:288] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06d7d5fb6584491faf26ab94a4341c39 (127.9.44.66:33305), ca66f7e2cb6e47a1a4820f72a41e40ec (127.9.44.67:44281)
W20260808 23:30:14.192088 32756 leader_election.cc:341] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca66f7e2cb6e47a1a4820f72a41e40ec (127.9.44.67:44281): Not found: Tablet not found: e238657407494aa0b315cc71d08130e6
W20260808 23:30:14.192255 32754 leader_election.cc:341] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 06d7d5fb6584491faf26ab94a4341c39 (127.9.44.66:33305): Not found: Tablet not found: e238657407494aa0b315cc71d08130e6
I20260808 23:30:14.192418 32754 leader_election.cc:302] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d818e1e5401472f9f3b0a667fee06b7; no voters: 06d7d5fb6584491faf26ab94a4341c39, ca66f7e2cb6e47a1a4820f72a41e40ec
I20260808 23:30:14.192713 807 raft_consensus.cc:2726] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 23:30:14.395884 808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:14.395180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.66:43033)
0808 23:30:14.395606 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:14.395721 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":217,"threads_started":1}
I20260808 23:30:14.845499 811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:14.844849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.67:37253)
0808 23:30:14.845265 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:14.845346 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":220,"threads_started":1}
I20260808 23:30:14.954964 812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:14.954204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.65:33155)
0808 23:30:14.954710 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:14.954792 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":289,"threads_started":1}
I20260808 23:30:15.397393 813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:15.396695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.66:44675)
0808 23:30:15.397136 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:15.397213 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":202,"threads_started":1}
I20260808 23:30:15.847041 814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:15.846278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.67:34639)
0808 23:30:15.846719 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:15.846817 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":220,"threads_started":1}
I20260808 23:30:15.956563 815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:15.955916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.65:45881)
0808 23:30:15.956313 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:15.956400 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":204,"threads_started":1}
I20260808 23:30:16.398852 817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:16.398164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.66:43889)
0808 23:30:16.398621 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:16.398704 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":244,"threads_started":1}
I20260808 23:30:16.848711 818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:16.847956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.67:53853)
0808 23:30:16.848464 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:16.848551 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":278,"threads_started":1}
I20260808 23:30:16.901607 820 raft_consensus.cc:482] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:30:16.901785 820 raft_consensus.cc:504] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } } peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } }
I20260808 23:30:16.902387 820 leader_election.cc:288] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06d7d5fb6584491faf26ab94a4341c39 (127.9.44.66:33305), ca66f7e2cb6e47a1a4820f72a41e40ec (127.9.44.67:44281)
W20260808 23:30:16.903264 32754 leader_election.cc:341] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 06d7d5fb6584491faf26ab94a4341c39 (127.9.44.66:33305): Not found: Tablet not found: e238657407494aa0b315cc71d08130e6
W20260808 23:30:16.903438 32756 leader_election.cc:341] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca66f7e2cb6e47a1a4820f72a41e40ec (127.9.44.67:44281): Not found: Tablet not found: e238657407494aa0b315cc71d08130e6
I20260808 23:30:16.903578 32756 leader_election.cc:302] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d818e1e5401472f9f3b0a667fee06b7; no voters: 06d7d5fb6584491faf26ab94a4341c39, ca66f7e2cb6e47a1a4820f72a41e40ec
I20260808 23:30:16.903851 820 raft_consensus.cc:2726] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 23:30:16.957988 821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:16.957286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.65:33179)
0808 23:30:16.957724 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:16.957814 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":218,"threads_started":1}
I20260808 23:30:17.400362 822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:17.399621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.66:42163)
0808 23:30:17.400110 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:17.400188 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":282,"threads_started":1}
I20260808 23:30:17.850203 823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:17.849468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.67:41509)
0808 23:30:17.849947 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:17.850032 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":245,"threads_started":1}
I20260808 23:30:17.959398 825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:17.958731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.65:37891)
0808 23:30:17.959127 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:17.959204 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":207,"threads_started":1}
I20260808 23:30:18.401875 827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:18.401138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.66:44851)
0808 23:30:18.401599 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:18.401678 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":201,"threads_started":1}
I20260808 23:30:18.851962 828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:18.851062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.67:43489)
0808 23:30:18.851631 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:18.851743 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":274,"threads_started":1}
I20260808 23:30:18.960963 829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:18.960242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.65:50391)
0808 23:30:18.960699 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:18.960801 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":238,"threads_started":1}
I20260808 23:30:19.332866 830 raft_consensus.cc:482] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:30:19.333086 830 raft_consensus.cc:504] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } } peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } }
I20260808 23:30:19.333701 830 leader_election.cc:288] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06d7d5fb6584491faf26ab94a4341c39 (127.9.44.66:33305), ca66f7e2cb6e47a1a4820f72a41e40ec (127.9.44.67:44281)
W20260808 23:30:19.334718 32754 leader_election.cc:341] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 06d7d5fb6584491faf26ab94a4341c39 (127.9.44.66:33305): Not found: Tablet not found: e238657407494aa0b315cc71d08130e6
W20260808 23:30:19.334882 32756 leader_election.cc:341] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca66f7e2cb6e47a1a4820f72a41e40ec (127.9.44.67:44281): Not found: Tablet not found: e238657407494aa0b315cc71d08130e6
I20260808 23:30:19.335036 32756 leader_election.cc:302] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d818e1e5401472f9f3b0a667fee06b7; no voters: 06d7d5fb6584491faf26ab94a4341c39, ca66f7e2cb6e47a1a4820f72a41e40ec
I20260808 23:30:19.335290 830 raft_consensus.cc:2726] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 23:30:19.403537 832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:19.402799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.66:36827)
0808 23:30:19.403245 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:19.403335 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":228,"threads_started":1}
I20260808 23:30:19.853375 833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:19.852742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.67:41677)
0808 23:30:19.853131 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:19.853209 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":191,"threads_started":1}
I20260808 23:30:19.962458 834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:19.961772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.65:47591)
0808 23:30:19.962193 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:19.962279 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":206,"threads_started":1}
I20260808 23:30:20.405084 835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:20.404378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.66:33913)
0808 23:30:20.404846 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:20.404927 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":214,"threads_started":1}
I20260808 23:30:20.854816 836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:20.854093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.67:34673)
0808 23:30:20.854573 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:20.854671 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":229,"threads_started":1}
I20260808 23:30:20.964092 837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:20.963308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.65:46515)
0808 23:30:20.963845 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:20.963932 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":283,"threads_started":1}
I20260808 23:30:21.375818 838 raft_consensus.cc:482] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:30:21.376013 838 raft_consensus.cc:504] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } } peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } }
I20260808 23:30:21.376698 838 leader_election.cc:288] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06d7d5fb6584491faf26ab94a4341c39 (127.9.44.66:33305), ca66f7e2cb6e47a1a4820f72a41e40ec (127.9.44.67:44281)
W20260808 23:30:21.377601 32756 leader_election.cc:341] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca66f7e2cb6e47a1a4820f72a41e40ec (127.9.44.67:44281): Not found: Tablet not found: e238657407494aa0b315cc71d08130e6
W20260808 23:30:21.377775 32754 leader_election.cc:341] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 06d7d5fb6584491faf26ab94a4341c39 (127.9.44.66:33305): Not found: Tablet not found: e238657407494aa0b315cc71d08130e6
I20260808 23:30:21.377930 32754 leader_election.cc:302] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d818e1e5401472f9f3b0a667fee06b7; no voters: 06d7d5fb6584491faf26ab94a4341c39, ca66f7e2cb6e47a1a4820f72a41e40ec
I20260808 23:30:21.378161 838 raft_consensus.cc:2726] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 23:30:21.406523 840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:21.405889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.66:57583)
0808 23:30:21.406272 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:21.406348 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":202,"threads_started":1}
I20260808 23:30:21.856268 841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:21.855566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.67:37445)
0808 23:30:21.856017 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:21.856094 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":253,"threads_started":1}
I20260808 23:30:21.965515 842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:21.964820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.65:42583)
0808 23:30:21.965249 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:21.965339 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":211,"threads_started":1}
I20260808 23:30:22.408028 843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:22.407315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.66:45729)
0808 23:30:22.407766 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:22.407861 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":206,"threads_started":1}
I20260808 23:30:22.857720 845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:22.857017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.67:34943)
0808 23:30:22.857473 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:22.857564 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":230,"threads_started":1}
I20260808 23:30:22.967047 846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:22.966305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.65:52027)
0808 23:30:22.966742 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:22.966821 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":203,"threads_started":1}
I20260808 23:30:23.409570 847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:23.408852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.66:34801)
0808 23:30:23.409312 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:23.409391 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":245,"threads_started":1}
I20260808 23:30:23.859206 848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:23.858520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.67:33701)
0808 23:30:23.858957 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:23.859036 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":223,"threads_started":1}
I20260808 23:30:23.968552 849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:23.967844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.65:53251)
0808 23:30:23.968307 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:23.968386 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":213,"threads_started":1}
I20260808 23:30:24.411121 850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:24.410425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.66:55837)
0808 23:30:24.410850 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:24.410943 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":212,"threads_started":1}
I20260808 23:30:24.860723 851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:24.860027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.67:46691)
0808 23:30:24.860466 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:24.860554 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":230,"threads_started":1}
I20260808 23:30:24.969962 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:24.969288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.65:36855)
0808 23:30:24.969699 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:24.969788 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":215,"threads_started":1}
I20260808 23:30:25.412647 853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:25.411907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.66:57245)
0808 23:30:25.412350 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:25.412433 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":226,"threads_started":1}
I20260808 23:30:25.754725 854 raft_consensus.cc:482] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:30:25.754925 854 raft_consensus.cc:504] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d818e1e5401472f9f3b0a667fee06b7" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 44937 } } peers { permanent_uuid: "06d7d5fb6584491faf26ab94a4341c39" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 33305 } } peers { permanent_uuid: "ca66f7e2cb6e47a1a4820f72a41e40ec" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44281 } }
I20260808 23:30:25.755602 854 leader_election.cc:288] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 06d7d5fb6584491faf26ab94a4341c39 (127.9.44.66:33305), ca66f7e2cb6e47a1a4820f72a41e40ec (127.9.44.67:44281)
W20260808 23:30:25.756564 32756 leader_election.cc:341] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca66f7e2cb6e47a1a4820f72a41e40ec (127.9.44.67:44281): Not found: Tablet not found: e238657407494aa0b315cc71d08130e6
W20260808 23:30:25.756729 32754 leader_election.cc:341] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 06d7d5fb6584491faf26ab94a4341c39 (127.9.44.66:33305): Not found: Tablet not found: e238657407494aa0b315cc71d08130e6
I20260808 23:30:25.756873 32754 leader_election.cc:302] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d818e1e5401472f9f3b0a667fee06b7; no voters: 06d7d5fb6584491faf26ab94a4341c39, ca66f7e2cb6e47a1a4820f72a41e40ec
I20260808 23:30:25.757102 854 raft_consensus.cc:2726] T e238657407494aa0b315cc71d08130e6 P 7d818e1e5401472f9f3b0a667fee06b7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 23:30:25.862201 856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:25.861454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.67:43045)
0808 23:30:25.861911 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:25.861988 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":241,"threads_started":1}
I20260808 23:30:25.971745 857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:25.970970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.65:60781)
0808 23:30:25.971424 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:25.971518 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":221,"threads_started":1}
I20260808 23:30:26.414324 859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:26.413637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.66:42797)
0808 23:30:26.414061 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:26.414146 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":215,"threads_started":1}
I20260808 23:30:26.838646 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 32674
I20260808 23:30:26.852607 861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:26.839725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57574 (local address 127.9.44.65:44937)
0808 23:30:26.840332 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:26.840339 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:26.840365 (+ 26us) server_negotiation.cc:407] Connection header received
0808 23:30:26.840426 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:26.840432 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:26.840652 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:26.840796 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:26.842472 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:26.843417 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:26.844624 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:26.845164 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:26.845260 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:26.847631 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:26.847643 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:26.847657 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:26.847721 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:26.850295 (+ 2574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:26.850923 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:26.850930 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:26.850932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:26.851064 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:26.851360 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:26.851363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:26.851365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:26.851719 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:26.851981 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:26.852142 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 23:30:26.852334 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":338,"threads_started":1}
I20260808 23:30:26.853835 422 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:30:26.864012 863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:26.863275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:37873 (local address 127.9.44.67:60511)
0808 23:30:26.863773 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:26.863859 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:37873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":247,"threads_started":1}
I20260808 23:30:26.940850 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 32674
I20260808 23:30:26.965380 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 436
I20260808 23:30:26.978157 864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:26.966179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57090 (local address 127.9.44.66:33305)
0808 23:30:26.966650 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:26.966654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:26.966671 (+ 17us) server_negotiation.cc:407] Connection header received
0808 23:30:26.966725 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:26.966729 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:26.966786 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:26.966874 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:26.968230 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:26.968977 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:26.970196 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:26.970618 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:26.970713 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:26.973257 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:26.973263 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:26.973266 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:26.973297 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:26.975886 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:26.976518 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:26.976523 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:26.976525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:26.976658 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:26.976952 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:26.976955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:26.976957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:26.977236 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:26.977426 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:26.977647 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 23:30:26.977806 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":243,"threads_started":1}
I20260808 23:30:26.978865 608 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:30:27.059649 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 436
I20260808 23:30:27.083997 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 618
I20260808 23:30:27.097007 866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:27.084845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52306 (local address 127.9.44.67:44281)
0808 23:30:27.085440 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:27.085445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:27.085463 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:30:27.085519 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:27.085523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:27.085585 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:27.085678 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:27.086948 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:27.087882 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:27.089063 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:27.089483 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:27.089574 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:27.092237 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:27.092242 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:27.092245 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:27.092278 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:27.094804 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:27.095443 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:27.095446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:27.095448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:27.095578 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:27.095899 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:27.095902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:27.095904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:27.096157 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:27.096386 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:27.096551 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 23:30:27.096708 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":291,"threads_started":1}
I20260808 23:30:27.097692 782 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:30:27.178459 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 618
2026-08-08T23:30:27Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24349 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:30:27 dist-test-slave-6bbx krb5kdc[872](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:30:27 dist-test-slave-6bbx krb5kdc[872](info): set up 2 sockets
Aug 08 23:30:27 dist-test-slave-6bbx krb5kdc[872](info): commencing operation
krb5kdc: starting...
W20260808 23:30:28.388286 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.005s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:30:28 dist-test-slave-6bbx krb5kdc[872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:30:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:30:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:30:28.624895 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:41945
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:41945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:30:28.943861 890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:30:28.944118 890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:30:28.944253 890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:30:28.951121 890 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 23:30:28.951213 890 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 23:30:28.951275 890 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 23:30:28.951346 890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:30:28.951417 890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:30:28.951512 890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:30:28.951572 890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:30:28.951654 890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:30:28.961030 890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:41945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:41945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:30:28.962133 890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:30:28.963366 890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:30:28.974184 897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:28.974292 904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:28.974319 899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:30:28.975381 890 server_base.cc:1034] running on GCE node
I20260808 23:30:28.976114 890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:30:28.977298 890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:30:28.978506 890 hybrid_clock.cc:648] HybridClock initialized: now 1786231828978449 us; error 28 us; skew 500 ppm
I20260808 23:30:28.978799 890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:30:28 dist-test-slave-6bbx krb5kdc[872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231828, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:30:28.988685 890 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:30:28.990036 890 webserver.cc:460] Webserver started at http://127.9.44.126:39343/ using document root <none> and password file <none>
I20260808 23:30:28.990670 890 fs_manager.cc:359] Metadata directory not provided
I20260808 23:30:28.990774 890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:30:28.991061 890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:30:28.993883 890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "20626351a0a84f9688ac89ec3cd9d37b"
format_stamp: "Formatted at 2026-08-08 23:30:28 on dist-test-slave-6bbx"
server_key: "0974298a5dbd7cea4a3249795db8dfdc"
server_key_iv: "1ea1c7a961da6a5d1371eccfd8695962"
server_key_version: "encryptionkey@0"
I20260808 23:30:28.994608 890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "20626351a0a84f9688ac89ec3cd9d37b"
format_stamp: "Formatted at 2026-08-08 23:30:28 on dist-test-slave-6bbx"
server_key: "0974298a5dbd7cea4a3249795db8dfdc"
server_key_iv: "1ea1c7a961da6a5d1371eccfd8695962"
server_key_version: "encryptionkey@0"
I20260808 23:30:28.999302 890 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.007s
I20260808 23:30:29.002667 913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:29.003917 890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 23:30:29.004062 890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "20626351a0a84f9688ac89ec3cd9d37b"
format_stamp: "Formatted at 2026-08-08 23:30:28 on dist-test-slave-6bbx"
server_key: "0974298a5dbd7cea4a3249795db8dfdc"
server_key_iv: "1ea1c7a961da6a5d1371eccfd8695962"
server_key_version: "encryptionkey@0"
I20260808 23:30:29.004268 890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:30:29.022830 890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:30:29.026410 890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:30:29.026640 890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:30:29.043571 890 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:41945
I20260808 23:30:29.043646 994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:41945 every 8 connection(s)
I20260808 23:30:29.044816 890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:30:29.048288 999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:30:29.053846 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 890
I20260808 23:30:29.054077 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:30:29.054906 9393 external_mini_cluster.cc:1428] Setting key 235e03a0779756c0601863537792f5f6
I20260808 23:30:29.059026 999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b: Bootstrap starting.
I20260808 23:30:29.062501 999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b: Neither blocks nor log segments found. Creating new log.
I20260808 23:30:29.063611 999 log.cc:824] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b: Log is configured to *not* fsync() on all Append() calls
I20260808 23:30:29.066444 999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b: No bootstrap required, opened a new log
Aug 08 23:30:29 dist-test-slave-6bbx krb5kdc[872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:30:29.074435 999 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20626351a0a84f9688ac89ec3cd9d37b" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 41945 } }
I20260808 23:30:29.074867 999 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:30:29.074987 999 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20626351a0a84f9688ac89ec3cd9d37b, State: Initialized, Role: FOLLOWER
I20260808 23:30:29.075682 999 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20626351a0a84f9688ac89ec3cd9d37b" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 41945 } }
I20260808 23:30:29.076094 999 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:30:29.076313 999 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:30:29.076531 999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:30:29.078078 1002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:29.056893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38614 (local address 127.9.44.126:41945)
0808 23:30:29.057610 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:29.057620 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:29.057657 (+ 37us) server_negotiation.cc:407] Connection header received
0808 23:30:29.059220 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:29.059246 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:29.059576 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:29.059914 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:29.061750 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:29.063083 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:29.064853 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:29.065481 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:29.065599 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:29.070185 (+ 4586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:29.070193 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:29.070206 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:29.070248 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:29.073186 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:29.073825 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:29.073831 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:29.073836 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:29.074000 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:29.074397 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:29.074401 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:29.074404 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:29.075099 (+ 695us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:29.075438 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:29.076248 (+ 810us) server_negotiation.cc:299] Negotiation successful
0808 23:30:29.076725 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":337,"threads_started":1}
I20260808 23:30:29.079722 999 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20626351a0a84f9688ac89ec3cd9d37b" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 41945 } }
I20260808 23:30:29.080302 999 leader_election.cc:302] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20626351a0a84f9688ac89ec3cd9d37b; no voters:
I20260808 23:30:29.081002 999 leader_election.cc:288] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:30:29.081365 1004 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:30:29.083573 1004 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [term 1 LEADER]: Becoming Leader. State: Replica: 20626351a0a84f9688ac89ec3cd9d37b, State: Running, Role: LEADER
I20260808 23:30:29.083931 999 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:30:29.084153 1004 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20626351a0a84f9688ac89ec3cd9d37b" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 41945 } }
I20260808 23:30:29.089169 1005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20626351a0a84f9688ac89ec3cd9d37b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20626351a0a84f9688ac89ec3cd9d37b" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 41945 } } }
I20260808 23:30:29.089659 1005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [sys.catalog]: This master's current role is: LEADER
I20260808 23:30:29.089622 1006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20626351a0a84f9688ac89ec3cd9d37b. Latest consensus state: current_term: 1 leader_uuid: "20626351a0a84f9688ac89ec3cd9d37b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20626351a0a84f9688ac89ec3cd9d37b" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 41945 } } }
I20260808 23:30:29.089929 1006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [sys.catalog]: This master's current role is: LEADER
W20260808 23:30:29.093498 1023 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:30:29.093597 1023 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:30:29.093716 1024 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:30:29.095299 1024 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:30:29.102798 1024 catalog_manager.cc:1347] Generated new cluster ID: f9646ad24674490abeab8d02016cc839
I20260808 23:30:29.102905 1024 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:30:29.128484 1024 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:30:29.129679 1024 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:30:29.134938 1024 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b: Generated new TSK 0
I20260808 23:30:29.135649 1024 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.44.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
I20260808 23:30:29.201282 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:0
--local_ip_for_outbound_sockets=127.9.44.65
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:41945
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:30:29.522264 1037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:30:29.522518 1037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:30:29.522663 1037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:30:29.529503 1037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:30:29.529615 1037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:30:29.529760 1037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:30:29.538722 1037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:41945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:30:29.539816 1037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:30:29.540999 1037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:30:29.552860 1052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:29.552893 1045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:29.552920 1048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:30:29.553929 1037 server_base.cc:1034] running on GCE node
I20260808 23:30:29.554477 1037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:30:29.555330 1037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:30:29.556521 1037 hybrid_clock.cc:648] HybridClock initialized: now 1786231829556478 us; error 29 us; skew 500 ppm
I20260808 23:30:29.556808 1037 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:30:29 dist-test-slave-6bbx krb5kdc[872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231829, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:30:29.559890 1037 init.cc:377] Logged in from keytab as kudu/127.9.44.65@KRBTEST.COM (short username kudu)
I20260808 23:30:29.561143 1037 webserver.cc:460] Webserver started at http://127.9.44.65:44233/ using document root <none> and password file <none>
I20260808 23:30:29.561741 1037 fs_manager.cc:359] Metadata directory not provided
I20260808 23:30:29.561839 1037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:30:29.562088 1037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:30:29.564782 1037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/data/instance:
uuid: "27834cae70bf4f74a636e853ad33a807"
format_stamp: "Formatted at 2026-08-08 23:30:29 on dist-test-slave-6bbx"
server_key: "706b1eb6906a4672ba3349a6a49d4432"
server_key_iv: "415485939e3d056a41c252a7ae6a8d50"
server_key_version: "encryptionkey@0"
I20260808 23:30:29.565471 1037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance:
uuid: "27834cae70bf4f74a636e853ad33a807"
format_stamp: "Formatted at 2026-08-08 23:30:29 on dist-test-slave-6bbx"
server_key: "706b1eb6906a4672ba3349a6a49d4432"
server_key_iv: "415485939e3d056a41c252a7ae6a8d50"
server_key_version: "encryptionkey@0"
I20260808 23:30:29.570233 1037 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 23:30:29.573385 1063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:29.574528 1037 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 23:30:29.574651 1037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "27834cae70bf4f74a636e853ad33a807"
format_stamp: "Formatted at 2026-08-08 23:30:29 on dist-test-slave-6bbx"
server_key: "706b1eb6906a4672ba3349a6a49d4432"
server_key_iv: "415485939e3d056a41c252a7ae6a8d50"
server_key_version: "encryptionkey@0"
I20260808 23:30:29.574810 1037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:30:29.589193 1037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:30:29.593039 1037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:30:29.593263 1037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:30:29.594069 1037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:30:29.595252 1037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:30:29.595320 1037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:29.595414 1037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:30:29.595460 1037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:29.623776 1037 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:40463
I20260808 23:30:29.623823 1225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:40463 every 8 connection(s)
I20260808 23:30:29.624919 1037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:30:29.635208 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 1037
I20260808 23:30:29.635397 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance
I20260808 23:30:29.636067 9393 external_mini_cluster.cc:1428] Setting key 5a41349cba406c589019638c8eb76e18
Aug 08 23:30:29 dist-test-slave-6bbx krb5kdc[872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231829, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:30:29.643168 1230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:29.627145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:36425 (local address 127.9.44.126:41945)
0808 23:30:29.627571 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:29.627576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:29.628397 (+ 821us) server_negotiation.cc:407] Connection header received
0808 23:30:29.630131 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:29.630138 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:29.630222 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:29.630325 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:29.631735 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:29.632459 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:29.633934 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:29.634465 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:29.634580 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:29.637997 (+ 3417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:29.638002 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:29.638005 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:29.638036 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:29.640406 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:29.641139 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:29.641142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:29.641145 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:29.641262 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:29.641671 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:29.641674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:29.641676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:29.641921 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:29.642026 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:29.642744 (+ 718us) server_negotiation.cc:299] Negotiation successful
0808 23:30:29.642918 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":225,"threads_started":1}
I20260808 23:30:29.644318 1231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:29.627506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.65:36425)
0808 23:30:29.628246 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:29.628282 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:29.629867 (+ 1585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:29.630556 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:29.630563 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:29.631016 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:30:29.631542 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:29.631559 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:29.632611 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:29.632615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:29.633791 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:29.633805 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:29.634694 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:29.634702 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:29.634930 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:29.635538 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:30:29.635561 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:30:29.637792 (+ 2231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:30:29.640539 (+ 2747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:29.640546 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:29.640560 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:29.641044 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:29.641351 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:29.641355 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:29.641359 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:29.641584 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:29.642031 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:30:29.642038 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:30:29.642410 (+ 372us) client_negotiation.cc:769] Sending connection context
0808 23:30:29.642672 (+ 262us) client_negotiation.cc:240] Negotiation successful
0808 23:30:29.643114 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":310,"threads_started":1}
I20260808 23:30:29.645784 1226 heartbeater.cc:347] Connected to a master server at 127.9.44.126:41945
I20260808 23:30:29.646044 1226 heartbeater.cc:464] Registering TS with master...
I20260808 23:30:29.646694 1226 heartbeater.cc:511] Master 127.9.44.126:41945 requested a full tablet report, sending...
I20260808 23:30:29.648648 941 ts_manager.cc:194] Registered new tserver with Master: 27834cae70bf4f74a636e853ad33a807 (127.9.44.65:40463)
I20260808 23:30:29.650009 941 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.65@KRBTEST.COM'} at 127.9.44.65:36425
WARNING: no policy specified for kudu/127.9.44.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
I20260808 23:30:29.694175 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:0
--local_ip_for_outbound_sockets=127.9.44.66
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:41945
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:30:30.007887 1235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:30:30.008144 1235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:30:30.008275 1235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:30:30.015110 1235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:30:30.015221 1235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:30:30.015372 1235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:30:30.024408 1235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:41945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:30:30.025503 1235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:30:30.026732 1235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:30:30.038916 1240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:30.038858 1243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:30.038944 1241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:30:30.039769 1235 server_base.cc:1034] running on GCE node
I20260808 23:30:30.040375 1235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:30:30.041200 1235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:30:30.042400 1235 hybrid_clock.cc:648] HybridClock initialized: now 1786231830042353 us; error 38 us; skew 500 ppm
I20260808 23:30:30.042687 1235 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:30:30 dist-test-slave-6bbx krb5kdc[872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231830, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:30:30.045722 1235 init.cc:377] Logged in from keytab as kudu/127.9.44.66@KRBTEST.COM (short username kudu)
I20260808 23:30:30.047142 1235 webserver.cc:460] Webserver started at http://127.9.44.66:45181/ using document root <none> and password file <none>
I20260808 23:30:30.047791 1235 fs_manager.cc:359] Metadata directory not provided
I20260808 23:30:30.047905 1235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:30:30.048197 1235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:30:30.050864 1235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/data/instance:
uuid: "db14e75d01c14755b969b34a0b3e4900"
format_stamp: "Formatted at 2026-08-08 23:30:30 on dist-test-slave-6bbx"
server_key: "1071a4017579a65def535a7cdc803bc6"
server_key_iv: "91d887173faaa0a4fdda8d5a7f11c016"
server_key_version: "encryptionkey@0"
I20260808 23:30:30.051571 1235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance:
uuid: "db14e75d01c14755b969b34a0b3e4900"
format_stamp: "Formatted at 2026-08-08 23:30:30 on dist-test-slave-6bbx"
server_key: "1071a4017579a65def535a7cdc803bc6"
server_key_iv: "91d887173faaa0a4fdda8d5a7f11c016"
server_key_version: "encryptionkey@0"
I20260808 23:30:30.056524 1235 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260808 23:30:30.059834 1254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:30.060990 1235 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 23:30:30.061123 1235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "db14e75d01c14755b969b34a0b3e4900"
format_stamp: "Formatted at 2026-08-08 23:30:30 on dist-test-slave-6bbx"
server_key: "1071a4017579a65def535a7cdc803bc6"
server_key_iv: "91d887173faaa0a4fdda8d5a7f11c016"
server_key_version: "encryptionkey@0"
I20260808 23:30:30.061285 1235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:30:30.085765 1235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:30:30.089664 1235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:30:30.089921 1235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:30:30.090777 1235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:30:30.092018 1235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:30:30.092089 1235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:30.092199 1235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:30:30.092252 1235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:30.119845 1235 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:38901
I20260808 23:30:30.119907 1403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:38901 every 8 connection(s)
I20260808 23:30:30.120986 1235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:30:30.125072 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 1235
I20260808 23:30:30.125264 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance
I20260808 23:30:30.125851 9393 external_mini_cluster.cc:1428] Setting key 3a5b8e2b5f538c77c5797056f6aa11ec
Aug 08 23:30:30 dist-test-slave-6bbx krb5kdc[872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231830, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:30:30.141260 1230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:30.123320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:46733 (local address 127.9.44.126:41945)
0808 23:30:30.123518 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:30.123525 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:30.124603 (+ 1078us) server_negotiation.cc:407] Connection header received
0808 23:30:30.126989 (+ 2386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:30.126995 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:30.127071 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:30.127170 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:30.128971 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.129681 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.131766 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.132150 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.132242 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:30.135902 (+ 3660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:30.135907 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:30.135910 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:30.135943 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:30.138411 (+ 2468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:30.139082 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:30.139086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:30.139088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:30.139210 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:30.139623 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:30.139627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:30.139629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:30.139928 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:30.140074 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:30.140721 (+ 647us) server_negotiation.cc:299] Negotiation successful
0808 23:30:30.140950 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 23:30:30.142076 1406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:30.123773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.66:46733)
0808 23:30:30.124441 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:30.124486 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:30.126665 (+ 2179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:30.127358 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:30.127368 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:30.128002 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:30:30.128784 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:30.128812 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.129840 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.129846 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:30.131592 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:30.131608 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.132363 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.132371 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:30.132624 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:30.133288 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:30:30.133315 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:30:30.135654 (+ 2339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:30:30.138542 (+ 2888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:30.138551 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:30.138566 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:30.138960 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:30.139323 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:30.139328 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:30.139332 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:30.139509 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:30.140056 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:30:30.140064 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:30:30.140364 (+ 300us) client_negotiation.cc:769] Sending connection context
0808 23:30:30.140584 (+ 220us) client_negotiation.cc:240] Negotiation successful
0808 23:30:30.141047 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":257,"threads_started":1}
I20260808 23:30:30.143478 1404 heartbeater.cc:347] Connected to a master server at 127.9.44.126:41945
I20260808 23:30:30.143776 1404 heartbeater.cc:464] Registering TS with master...
I20260808 23:30:30.144415 1404 heartbeater.cc:511] Master 127.9.44.126:41945 requested a full tablet report, sending...
I20260808 23:30:30.145999 941 ts_manager.cc:194] Registered new tserver with Master: db14e75d01c14755b969b34a0b3e4900 (127.9.44.66:38901)
I20260808 23:30:30.146755 941 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.66@KRBTEST.COM'} at 127.9.44.66:46733
WARNING: no policy specified for kudu/127.9.44.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
I20260808 23:30:30.183722 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:0
--local_ip_for_outbound_sockets=127.9.44.67
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:41945
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:30:30.508968 1409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:30:30.509222 1409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:30:30.509317 1409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:30:30.516362 1409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:30:30.516455 1409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:30:30.516569 1409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:30:30.525727 1409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:41945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:30:30.526808 1409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:30:30.528048 1409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:30:30.540841 1418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:30.540963 1417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:30.540803 1422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:30:30.541733 1409 server_base.cc:1034] running on GCE node
I20260808 23:30:30.542313 1409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:30:30.543149 1409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:30:30.544340 1409 hybrid_clock.cc:648] HybridClock initialized: now 1786231830544284 us; error 44 us; skew 500 ppm
I20260808 23:30:30.544627 1409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:30:30 dist-test-slave-6bbx krb5kdc[872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231830, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:30:30.547495 1409 init.cc:377] Logged in from keytab as kudu/127.9.44.67@KRBTEST.COM (short username kudu)
I20260808 23:30:30.548877 1409 webserver.cc:460] Webserver started at http://127.9.44.67:42609/ using document root <none> and password file <none>
I20260808 23:30:30.549508 1409 fs_manager.cc:359] Metadata directory not provided
I20260808 23:30:30.549618 1409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:30:30.549921 1409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:30:30.552699 1409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/data/instance:
uuid: "a951a5353a7e407981ce7ebc27ecabf4"
format_stamp: "Formatted at 2026-08-08 23:30:30 on dist-test-slave-6bbx"
server_key: "9e77796ec4a7a1180f15d98f6f1420df"
server_key_iv: "5c336e7edf9279c61ef79a4f89bb7b88"
server_key_version: "encryptionkey@0"
I20260808 23:30:30.553436 1409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance:
uuid: "a951a5353a7e407981ce7ebc27ecabf4"
format_stamp: "Formatted at 2026-08-08 23:30:30 on dist-test-slave-6bbx"
server_key: "9e77796ec4a7a1180f15d98f6f1420df"
server_key_iv: "5c336e7edf9279c61ef79a4f89bb7b88"
server_key_version: "encryptionkey@0"
I20260808 23:30:30.558351 1409 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 23:30:30.561815 1433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:30.563025 1409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:30:30.563179 1409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "a951a5353a7e407981ce7ebc27ecabf4"
format_stamp: "Formatted at 2026-08-08 23:30:30 on dist-test-slave-6bbx"
server_key: "9e77796ec4a7a1180f15d98f6f1420df"
server_key_iv: "5c336e7edf9279c61ef79a4f89bb7b88"
server_key_version: "encryptionkey@0"
I20260808 23:30:30.563367 1409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:30:30.579308 1409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:30:30.582870 1409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:30:30.583115 1409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:30:30.584031 1409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:30:30.585341 1409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:30:30.585413 1409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:30.585523 1409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:30:30.585577 1409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:30.615733 1409 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:37569
I20260808 23:30:30.615800 1582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:37569 every 8 connection(s)
I20260808 23:30:30.616889 1409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:30:30.625242 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 1409
I20260808 23:30:30.625439 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance
I20260808 23:30:30.626000 9393 external_mini_cluster.cc:1428] Setting key b45d5344ee8d8b32253ff3a5453e0af5
Aug 08 23:30:30 dist-test-slave-6bbx krb5kdc[872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231830, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:30:30.635196 1230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:30.619067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:41343 (local address 127.9.44.126:41945)
0808 23:30:30.619225 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:30.619229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:30.620311 (+ 1082us) server_negotiation.cc:407] Connection header received
0808 23:30:30.622083 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:30.622089 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:30.622218 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:30.622314 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:30.623843 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.624594 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.626248 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.626785 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.626905 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:30.630417 (+ 3512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:30.630421 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:30.630425 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:30.630458 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:30.632748 (+ 2290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:30.633377 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:30.633381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:30.633383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:30.633504 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:30.633835 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:30.633838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:30.633840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:30.634096 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:30.634201 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:30.634801 (+ 600us) server_negotiation.cc:299] Negotiation successful
0808 23:30:30.634961 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 23:30:30.635985 1585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:30.619440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.67:41343)
0808 23:30:30.620155 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:30.620197 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:30.621828 (+ 1631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:30.622563 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:30.622571 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:30.623040 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:30:30.623635 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:30.623653 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.624753 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.624757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:30.626092 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:30.626108 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.627008 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.627016 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:30.627287 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:30.627987 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:30:30.628009 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:30:30.630240 (+ 2231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:30:30.632869 (+ 2629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:30.632875 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:30.632885 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:30.633287 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:30.633577 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:30.633580 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:30.633582 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:30.633755 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:30.634186 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:30:30.634191 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:30:30.634471 (+ 280us) client_negotiation.cc:769] Sending connection context
0808 23:30:30.634676 (+ 205us) client_negotiation.cc:240] Negotiation successful
0808 23:30:30.634999 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":289,"threads_started":1}
I20260808 23:30:30.637162 1583 heartbeater.cc:347] Connected to a master server at 127.9.44.126:41945
I20260808 23:30:30.637391 1583 heartbeater.cc:464] Registering TS with master...
I20260808 23:30:30.638003 1583 heartbeater.cc:511] Master 127.9.44.126:41945 requested a full tablet report, sending...
I20260808 23:30:30.639246 941 ts_manager.cc:194] Registered new tserver with Master: a951a5353a7e407981ce7ebc27ecabf4 (127.9.44.67:37569)
I20260808 23:30:30.640163 941 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.67@KRBTEST.COM'} at 127.9.44.67:41343
I20260808 23:30:30.641523 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:30:30.652920 1226 heartbeater.cc:503] Master 127.9.44.126:41945 was elected leader, sending a full tablet report...
I20260808 23:30:30.658394 1230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:30.646151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38628 (local address 127.9.44.126:41945)
0808 23:30:30.646295 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:30.646299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:30.646720 (+ 421us) server_negotiation.cc:407] Connection header received
0808 23:30:30.646863 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:30.646868 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:30.646930 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:30.647006 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:30.648475 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.649144 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.650495 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.650894 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.650999 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:30.653418 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:30.653423 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:30.653426 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:30.653453 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:30.655969 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:30.656584 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:30.656590 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:30.656593 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:30.656744 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:30.657087 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:30.657092 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:30.657096 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:30.657381 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:30.657629 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:30.657915 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 23:30:30.658108 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 23:30:30.661907 941 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38628:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:30:30.668633 941 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:30:30.701596 1603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:30.680915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45798 (local address 127.9.44.65:40463)
0808 23:30:30.681594 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:30.681604 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:30.692624 (+ 11020us) server_negotiation.cc:407] Connection header received
0808 23:30:30.692739 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:30.692748 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:30.693016 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:30.693203 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:30:30.694138 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.695919 (+ 1781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.698548 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.700033 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.700141 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:30.700390 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:30.701094 (+ 704us) server_negotiation.cc:299] Negotiation successful
0808 23:30:30.701267 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":307,"threads_started":1}
I20260808 23:30:30.701705 1606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:30.680444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:40463 (local address 127.0.0.1:45798)
0808 23:30:30.692353 (+ 11909us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:30.692371 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:30.692499 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:30.693489 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:30.693498 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:30.693530 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:30:30.693932 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:30.693945 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.696114 (+ 2169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.696119 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:30.698362 (+ 2243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:30.698377 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.700548 (+ 2171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.700555 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:30.700778 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:30.700884 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 23:30:30.701033 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 23:30:30.701242 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11775,"mutex_wait_us":26,"thread_start_us":10906,"threads_started":1}
I20260808 23:30:30.704473 1600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:30.682792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:38901 (local address 127.0.0.1:47496)
0808 23:30:30.696959 (+ 14167us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:30.696981 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:30.697120 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:30.697691 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:30.697695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:30.697731 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:30:30.698159 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:30.698173 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.700161 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.700166 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:30.702253 (+ 2087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:30.702266 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.703586 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.703593 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:30.703829 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:30.703872 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 23:30:30.703973 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 23:30:30.704097 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14016,"mutex_wait_us":56,"thread_start_us":9261,"threads_started":1}
I20260808 23:30:30.704519 1607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:30.683536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47496 (local address 127.9.44.66:38901)
0808 23:30:30.690750 (+ 7214us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:30.690759 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:30.697098 (+ 6339us) server_negotiation.cc:407] Connection header received
0808 23:30:30.697257 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:30.697264 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:30.697511 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:30.697692 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:30:30.698387 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.699995 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.702403 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.703379 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.703518 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:30.703724 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:30.704030 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 23:30:30.704200 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7050,"thread_start_us":6934,"threads_started":1}
I20260808 23:30:30.705793 1601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:30.680306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:37569 (local address 127.0.0.1:55972)
0808 23:30:30.690942 (+ 10636us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:30.690991 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:30.691164 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:30.698250 (+ 7086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:30.698255 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:30.698281 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:30:30.698633 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:30.698642 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.700174 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.700179 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:30.702492 (+ 2313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:30.702509 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.705023 (+ 2514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.705033 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:30.705198 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:30.705230 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 23:30:30.705318 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 23:30:30.705450 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10380,"mutex_wait_us":60,"thread_start_us":16319,"threads_started":1}
I20260808 23:30:30.706478 1599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:30.680362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55972 (local address 127.9.44.67:37569)
0808 23:30:30.697694 (+ 17332us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:30.697702 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:30.697731 (+ 29us) server_negotiation.cc:407] Connection header received
0808 23:30:30.697823 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:30.697830 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:30.698042 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:30.698250 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:30:30.698792 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.700015 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.702678 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.703980 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.704084 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:30.704300 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:30.706117 (+ 1817us) server_negotiation.cc:299] Negotiation successful
0808 23:30:30.706246 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17171,"thread_start_us":17016,"threads_started":1}
I20260808 23:30:30.706136 1136 tablet_service.cc:1467] Processing CreateTablet for tablet 7524509a469e48ac81f30d88f7cabdb5 (DEFAULT_TABLE table=test-table [id=b6980a27840545f682bf211a2dad7cfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:30:30.707653 1136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7524509a469e48ac81f30d88f7cabdb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:30:30.708330 1315 tablet_service.cc:1467] Processing CreateTablet for tablet 7524509a469e48ac81f30d88f7cabdb5 (DEFAULT_TABLE table=test-table [id=b6980a27840545f682bf211a2dad7cfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:30:30.709682 1315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7524509a469e48ac81f30d88f7cabdb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:30:30.710340 1499 tablet_service.cc:1467] Processing CreateTablet for tablet 7524509a469e48ac81f30d88f7cabdb5 (DEFAULT_TABLE table=test-table [id=b6980a27840545f682bf211a2dad7cfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:30:30.711714 1499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7524509a469e48ac81f30d88f7cabdb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:30:30.721921 1608 tablet_bootstrap.cc:492] T 7524509a469e48ac81f30d88f7cabdb5 P a951a5353a7e407981ce7ebc27ecabf4: Bootstrap starting.
I20260808 23:30:30.723773 1611 tablet_bootstrap.cc:492] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900: Bootstrap starting.
I20260808 23:30:30.725715 1608 tablet_bootstrap.cc:654] T 7524509a469e48ac81f30d88f7cabdb5 P a951a5353a7e407981ce7ebc27ecabf4: Neither blocks nor log segments found. Creating new log.
I20260808 23:30:30.726235 1609 tablet_bootstrap.cc:492] T 7524509a469e48ac81f30d88f7cabdb5 P 27834cae70bf4f74a636e853ad33a807: Bootstrap starting.
I20260808 23:30:30.727183 1611 tablet_bootstrap.cc:654] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900: Neither blocks nor log segments found. Creating new log.
I20260808 23:30:30.728305 1608 log.cc:824] T 7524509a469e48ac81f30d88f7cabdb5 P a951a5353a7e407981ce7ebc27ecabf4: Log is configured to *not* fsync() on all Append() calls
I20260808 23:30:30.728377 1611 log.cc:824] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900: Log is configured to *not* fsync() on all Append() calls
I20260808 23:30:30.729138 1609 tablet_bootstrap.cc:654] T 7524509a469e48ac81f30d88f7cabdb5 P 27834cae70bf4f74a636e853ad33a807: Neither blocks nor log segments found. Creating new log.
I20260808 23:30:30.730221 1609 log.cc:824] T 7524509a469e48ac81f30d88f7cabdb5 P 27834cae70bf4f74a636e853ad33a807: Log is configured to *not* fsync() on all Append() calls
I20260808 23:30:30.731495 1611 tablet_bootstrap.cc:492] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900: No bootstrap required, opened a new log
I20260808 23:30:30.731496 1608 tablet_bootstrap.cc:492] T 7524509a469e48ac81f30d88f7cabdb5 P a951a5353a7e407981ce7ebc27ecabf4: No bootstrap required, opened a new log
I20260808 23:30:30.731796 1611 ts_tablet_manager.cc:1397] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260808 23:30:30.731796 1608 ts_tablet_manager.cc:1397] T 7524509a469e48ac81f30d88f7cabdb5 P a951a5353a7e407981ce7ebc27ecabf4: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260808 23:30:30.732548 1609 tablet_bootstrap.cc:492] T 7524509a469e48ac81f30d88f7cabdb5 P 27834cae70bf4f74a636e853ad33a807: No bootstrap required, opened a new log
I20260808 23:30:30.732781 1609 ts_tablet_manager.cc:1397] T 7524509a469e48ac81f30d88f7cabdb5 P 27834cae70bf4f74a636e853ad33a807: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 23:30:30.740011 1609 raft_consensus.cc:348] T 7524509a469e48ac81f30d88f7cabdb5 P 27834cae70bf4f74a636e853ad33a807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a951a5353a7e407981ce7ebc27ecabf4" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 37569 } } peers { permanent_uuid: "27834cae70bf4f74a636e853ad33a807" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40463 } } peers { permanent_uuid: "db14e75d01c14755b969b34a0b3e4900" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38901 } }
I20260808 23:30:30.740365 1609 raft_consensus.cc:374] T 7524509a469e48ac81f30d88f7cabdb5 P 27834cae70bf4f74a636e853ad33a807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:30:30.740463 1609 raft_consensus.cc:729] T 7524509a469e48ac81f30d88f7cabdb5 P 27834cae70bf4f74a636e853ad33a807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27834cae70bf4f74a636e853ad33a807, State: Initialized, Role: FOLLOWER
I20260808 23:30:30.741004 1609 consensus_queue.cc:260] T 7524509a469e48ac81f30d88f7cabdb5 P 27834cae70bf4f74a636e853ad33a807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a951a5353a7e407981ce7ebc27ecabf4" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 37569 } } peers { permanent_uuid: "27834cae70bf4f74a636e853ad33a807" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40463 } } peers { permanent_uuid: "db14e75d01c14755b969b34a0b3e4900" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38901 } }
I20260808 23:30:30.741981 1608 raft_consensus.cc:348] T 7524509a469e48ac81f30d88f7cabdb5 P a951a5353a7e407981ce7ebc27ecabf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a951a5353a7e407981ce7ebc27ecabf4" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 37569 } } peers { permanent_uuid: "27834cae70bf4f74a636e853ad33a807" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40463 } } peers { permanent_uuid: "db14e75d01c14755b969b34a0b3e4900" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38901 } }
I20260808 23:30:30.741917 1611 raft_consensus.cc:348] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a951a5353a7e407981ce7ebc27ecabf4" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 37569 } } peers { permanent_uuid: "27834cae70bf4f74a636e853ad33a807" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40463 } } peers { permanent_uuid: "db14e75d01c14755b969b34a0b3e4900" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38901 } }
I20260808 23:30:30.742378 1608 raft_consensus.cc:374] T 7524509a469e48ac81f30d88f7cabdb5 P a951a5353a7e407981ce7ebc27ecabf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:30:30.742389 1611 raft_consensus.cc:374] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:30:30.742473 1608 raft_consensus.cc:729] T 7524509a469e48ac81f30d88f7cabdb5 P a951a5353a7e407981ce7ebc27ecabf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a951a5353a7e407981ce7ebc27ecabf4, State: Initialized, Role: FOLLOWER
I20260808 23:30:30.742472 1611 raft_consensus.cc:729] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db14e75d01c14755b969b34a0b3e4900, State: Initialized, Role: FOLLOWER
I20260808 23:30:30.742625 1609 ts_tablet_manager.cc:1428] T 7524509a469e48ac81f30d88f7cabdb5 P 27834cae70bf4f74a636e853ad33a807: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260808 23:30:30.743139 1611 consensus_queue.cc:260] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a951a5353a7e407981ce7ebc27ecabf4" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 37569 } } peers { permanent_uuid: "27834cae70bf4f74a636e853ad33a807" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40463 } } peers { permanent_uuid: "db14e75d01c14755b969b34a0b3e4900" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38901 } }
I20260808 23:30:30.743180 1608 consensus_queue.cc:260] T 7524509a469e48ac81f30d88f7cabdb5 P a951a5353a7e407981ce7ebc27ecabf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a951a5353a7e407981ce7ebc27ecabf4" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 37569 } } peers { permanent_uuid: "27834cae70bf4f74a636e853ad33a807" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40463 } } peers { permanent_uuid: "db14e75d01c14755b969b34a0b3e4900" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38901 } }
I20260808 23:30:30.744787 1404 heartbeater.cc:503] Master 127.9.44.126:41945 was elected leader, sending a full tablet report...
I20260808 23:30:30.744880 1611 ts_tablet_manager.cc:1428] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900: Time spent starting tablet: real 0.013s user 0.007s sys 0.005s
I20260808 23:30:30.745247 1608 ts_tablet_manager.cc:1428] T 7524509a469e48ac81f30d88f7cabdb5 P a951a5353a7e407981ce7ebc27ecabf4: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260808 23:30:30.745615 1583 heartbeater.cc:503] Master 127.9.44.126:41945 was elected leader, sending a full tablet report...
W20260808 23:30:30.868465 1584 tablet.cc:2369] T 7524509a469e48ac81f30d88f7cabdb5 P a951a5353a7e407981ce7ebc27ecabf4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:30:30.872692 1405 tablet.cc:2369] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:30:30.876986 1228 tablet.cc:2369] T 7524509a469e48ac81f30d88f7cabdb5 P 27834cae70bf4f74a636e853ad33a807: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:30:30.952665 1616 raft_consensus.cc:482] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:30:30.952952 1616 raft_consensus.cc:504] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a951a5353a7e407981ce7ebc27ecabf4" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 37569 } } peers { permanent_uuid: "27834cae70bf4f74a636e853ad33a807" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40463 } } peers { permanent_uuid: "db14e75d01c14755b969b34a0b3e4900" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38901 } }
I20260808 23:30:30.954629 1616 leader_election.cc:288] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a951a5353a7e407981ce7ebc27ecabf4 (127.9.44.67:37569), 27834cae70bf4f74a636e853ad33a807 (127.9.44.65:40463)
I20260808 23:30:30.961138 1599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:30.955028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:49763 (local address 127.9.44.67:37569)
0808 23:30:30.955260 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:30.955267 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:30.955759 (+ 492us) server_negotiation.cc:407] Connection header received
0808 23:30:30.955987 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:30.955991 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:30.956052 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:30.956123 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:30:30.956701 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.957423 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.959274 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.960180 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.960296 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:30.960414 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:30.960768 (+ 354us) server_negotiation.cc:299] Negotiation successful
0808 23:30:30.960897 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:30:30.961508 1620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:30.954894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:37569 (local address 127.9.44.66:49763)
0808 23:30:30.955634 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:30.955650 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:30.955842 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:30.956166 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:30.956170 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:30.956195 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:30:30.956560 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:30.956571 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.957564 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.957569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:30.959118 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:30.959135 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.960421 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.960428 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:30.960588 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:30.960642 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 23:30:30.960727 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 23:30:30.960848 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"mutex_wait_us":51,"thread_start_us":380,"threads_started":1}
I20260808 23:30:30.961529 1621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:30.954903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:40463 (local address 127.9.44.66:50179)
0808 23:30:30.955940 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:30.955954 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:30.956077 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:30.956374 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:30.956378 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:30.956401 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:30:30.956763 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:30.956772 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.957738 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.957742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:30.959180 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:30.959193 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.960458 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.960465 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:30.960640 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:30.960664 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 23:30:30.960744 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 23:30:30.960862 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":939,"mutex_wait_us":219,"thread_start_us":530,"threads_started":1}
I20260808 23:30:30.962234 1603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:30.955035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:50179 (local address 127.9.44.65:40463)
0808 23:30:30.955440 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:30.955446 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:30.956039 (+ 593us) server_negotiation.cc:407] Connection header received
0808 23:30:30.956200 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:30.956204 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:30.956263 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:30.956334 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:30:30.956913 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.957599 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.959336 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:30.960222 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:30.960350 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:30.960448 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:30.960785 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 23:30:30.960902 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258}
I20260808 23:30:30.962690 1521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7524509a469e48ac81f30d88f7cabdb5" candidate_uuid: "db14e75d01c14755b969b34a0b3e4900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a951a5353a7e407981ce7ebc27ecabf4" is_pre_election: true
I20260808 23:30:30.962988 1162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7524509a469e48ac81f30d88f7cabdb5" candidate_uuid: "db14e75d01c14755b969b34a0b3e4900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27834cae70bf4f74a636e853ad33a807" is_pre_election: true
I20260808 23:30:30.963169 1521 raft_consensus.cc:2454] T 7524509a469e48ac81f30d88f7cabdb5 P a951a5353a7e407981ce7ebc27ecabf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db14e75d01c14755b969b34a0b3e4900 in term 0.
I20260808 23:30:30.963467 1162 raft_consensus.cc:2454] T 7524509a469e48ac81f30d88f7cabdb5 P 27834cae70bf4f74a636e853ad33a807 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db14e75d01c14755b969b34a0b3e4900 in term 0.
I20260808 23:30:30.963824 1257 leader_election.cc:302] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a951a5353a7e407981ce7ebc27ecabf4, db14e75d01c14755b969b34a0b3e4900; no voters:
I20260808 23:30:30.964195 1616 raft_consensus.cc:2781] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:30:30.964334 1616 raft_consensus.cc:482] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:30:30.964430 1616 raft_consensus.cc:3037] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:30:30.966624 1616 raft_consensus.cc:504] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a951a5353a7e407981ce7ebc27ecabf4" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 37569 } } peers { permanent_uuid: "27834cae70bf4f74a636e853ad33a807" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40463 } } peers { permanent_uuid: "db14e75d01c14755b969b34a0b3e4900" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38901 } }
I20260808 23:30:30.967172 1616 leader_election.cc:288] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900 [CANDIDATE]: Term 1 election: Requested vote from peers a951a5353a7e407981ce7ebc27ecabf4 (127.9.44.67:37569), 27834cae70bf4f74a636e853ad33a807 (127.9.44.65:40463)
I20260808 23:30:30.967521 1521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7524509a469e48ac81f30d88f7cabdb5" candidate_uuid: "db14e75d01c14755b969b34a0b3e4900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a951a5353a7e407981ce7ebc27ecabf4"
I20260808 23:30:30.967636 1162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7524509a469e48ac81f30d88f7cabdb5" candidate_uuid: "db14e75d01c14755b969b34a0b3e4900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27834cae70bf4f74a636e853ad33a807"
I20260808 23:30:30.967725 1521 raft_consensus.cc:3037] T 7524509a469e48ac81f30d88f7cabdb5 P a951a5353a7e407981ce7ebc27ecabf4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:30:30.967849 1162 raft_consensus.cc:3037] T 7524509a469e48ac81f30d88f7cabdb5 P 27834cae70bf4f74a636e853ad33a807 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:30:30.969949 1521 raft_consensus.cc:2454] T 7524509a469e48ac81f30d88f7cabdb5 P a951a5353a7e407981ce7ebc27ecabf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db14e75d01c14755b969b34a0b3e4900 in term 1.
I20260808 23:30:30.970007 1162 raft_consensus.cc:2454] T 7524509a469e48ac81f30d88f7cabdb5 P 27834cae70bf4f74a636e853ad33a807 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db14e75d01c14755b969b34a0b3e4900 in term 1.
I20260808 23:30:30.970299 1257 leader_election.cc:302] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a951a5353a7e407981ce7ebc27ecabf4, db14e75d01c14755b969b34a0b3e4900; no voters:
I20260808 23:30:30.970572 1616 raft_consensus.cc:2781] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:30:30.970984 1616 raft_consensus.cc:686] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900 [term 1 LEADER]: Becoming Leader. State: Replica: db14e75d01c14755b969b34a0b3e4900, State: Running, Role: LEADER
I20260808 23:30:30.971396 1616 consensus_queue.cc:237] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a951a5353a7e407981ce7ebc27ecabf4" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 37569 } } peers { permanent_uuid: "27834cae70bf4f74a636e853ad33a807" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40463 } } peers { permanent_uuid: "db14e75d01c14755b969b34a0b3e4900" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38901 } }
I20260808 23:30:30.976110 936 catalog_manager.cc:5495] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900 reported cstate change: term changed from 0 to 1, leader changed from <none> to db14e75d01c14755b969b34a0b3e4900 (127.9.44.66). New cstate: current_term: 1 leader_uuid: "db14e75d01c14755b969b34a0b3e4900" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a951a5353a7e407981ce7ebc27ecabf4" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 37569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27834cae70bf4f74a636e853ad33a807" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db14e75d01c14755b969b34a0b3e4900" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38901 } health_report { overall_health: HEALTHY } } }
I20260808 23:30:31.059926 1607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:31.054381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47502 (local address 127.9.44.66:38901)
0808 23:30:31.054577 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:31.054583 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:31.054710 (+ 127us) server_negotiation.cc:407] Connection header received
0808 23:30:31.054954 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:31.054958 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:31.055039 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:31.055174 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:30:31.055562 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:31.056432 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:31.057808 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:31.058214 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:31.058324 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:31.058809 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:30:31.059299 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:30:31.059455 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:31.059544 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 23:30:31.059628 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 23:30:31.068087 1162 raft_consensus.cc:1261] T 7524509a469e48ac81f30d88f7cabdb5 P 27834cae70bf4f74a636e853ad33a807 [term 1 FOLLOWER]: Refusing update from remote peer db14e75d01c14755b969b34a0b3e4900: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:30:31.068089 1521 raft_consensus.cc:1261] T 7524509a469e48ac81f30d88f7cabdb5 P a951a5353a7e407981ce7ebc27ecabf4 [term 1 FOLLOWER]: Refusing update from remote peer db14e75d01c14755b969b34a0b3e4900: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:30:31.069151 1616 consensus_queue.cc:1035] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a951a5353a7e407981ce7ebc27ecabf4" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 37569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:30:31.069469 1622 consensus_queue.cc:1035] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27834cae70bf4f74a636e853ad33a807" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:30:31.080399 1626 mvcc.cc:205] Tried to move back new op lower bound from 7316405580044460032 to 7316405579662254080. Current Snapshot: MvccSnapshot[applied={T|T < 7316405580044460032}]
I20260808 23:30:31.085040 1628 mvcc.cc:205] Tried to move back new op lower bound from 7316405580044460032 to 7316405579662254080. Current Snapshot: MvccSnapshot[applied={T|T < 7316405580044460032}]
I20260808 23:30:31.095391 936 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:30:31.095626 936 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:30:31.099099 936 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b: Sending DeleteTablet for 3 replicas of tablet 7524509a469e48ac81f30d88f7cabdb5
I20260808 23:30:31.100194 1136 tablet_service.cc:1514] Processing DeleteTablet for tablet 7524509a469e48ac81f30d88f7cabdb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:30:31 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:45798
I20260808 23:30:31.100310 1315 tablet_service.cc:1514] Processing DeleteTablet for tablet 7524509a469e48ac81f30d88f7cabdb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:30:31 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:47496
I20260808 23:30:31.100489 1499 tablet_service.cc:1514] Processing DeleteTablet for tablet 7524509a469e48ac81f30d88f7cabdb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:30:31 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:55972
I20260808 23:30:31.101052 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 1037
I20260808 23:30:31.115916 1642 tablet_replica.cc:330] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900: stopping tablet replica
I20260808 23:30:31.116288 1642 raft_consensus.cc:2229] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:30:31.116819 1642 raft_consensus.cc:2258] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:30:31.124819 1642 ts_tablet_manager.cc:1905] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:30:31.128218 1643 tablet_replica.cc:330] T 7524509a469e48ac81f30d88f7cabdb5 P a951a5353a7e407981ce7ebc27ecabf4: stopping tablet replica
I20260808 23:30:31.128628 1643 raft_consensus.cc:2229] T 7524509a469e48ac81f30d88f7cabdb5 P a951a5353a7e407981ce7ebc27ecabf4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:30:31.129136 1643 raft_consensus.cc:2258] T 7524509a469e48ac81f30d88f7cabdb5 P a951a5353a7e407981ce7ebc27ecabf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:30:31.129841 1641 tablet_replica.cc:330] T 7524509a469e48ac81f30d88f7cabdb5 P 27834cae70bf4f74a636e853ad33a807: stopping tablet replica
I20260808 23:30:31.130220 1641 raft_consensus.cc:2229] T 7524509a469e48ac81f30d88f7cabdb5 P 27834cae70bf4f74a636e853ad33a807 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:30:31.130715 1641 raft_consensus.cc:2258] T 7524509a469e48ac81f30d88f7cabdb5 P 27834cae70bf4f74a636e853ad33a807 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:30:31.134392 1642 ts_tablet_manager.cc:1918] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:30:31.134613 1642 log.cc:1196] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/7524509a469e48ac81f30d88f7cabdb5
I20260808 23:30:31.135226 1642 ts_tablet_manager.cc:1939] T 7524509a469e48ac81f30d88f7cabdb5 P db14e75d01c14755b969b34a0b3e4900: Deleting consensus metadata
I20260808 23:30:31.137022 915 catalog_manager.cc:4841] TS db14e75d01c14755b969b34a0b3e4900 (127.9.44.66:38901): tablet 7524509a469e48ac81f30d88f7cabdb5 (table test-table [id=b6980a27840545f682bf211a2dad7cfb]) successfully deleted
I20260808 23:30:31.137777 1641 ts_tablet_manager.cc:1905] T 7524509a469e48ac81f30d88f7cabdb5 P 27834cae70bf4f74a636e853ad33a807: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:30:31.137995 1643 ts_tablet_manager.cc:1905] T 7524509a469e48ac81f30d88f7cabdb5 P a951a5353a7e407981ce7ebc27ecabf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:30:31.144963 1643 ts_tablet_manager.cc:1918] T 7524509a469e48ac81f30d88f7cabdb5 P a951a5353a7e407981ce7ebc27ecabf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:30:31.145121 1643 log.cc:1196] T 7524509a469e48ac81f30d88f7cabdb5 P a951a5353a7e407981ce7ebc27ecabf4: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/7524509a469e48ac81f30d88f7cabdb5
I20260808 23:30:31.145651 1643 ts_tablet_manager.cc:1939] T 7524509a469e48ac81f30d88f7cabdb5 P a951a5353a7e407981ce7ebc27ecabf4: Deleting consensus metadata
Aug 08 23:30:31 dist-test-slave-6bbx krb5kdc[872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.65@KRBTEST.COM
I20260808 23:30:31.147439 916 catalog_manager.cc:4841] TS a951a5353a7e407981ce7ebc27ecabf4 (127.9.44.67:37569): tablet 7524509a469e48ac81f30d88f7cabdb5 (table test-table [id=b6980a27840545f682bf211a2dad7cfb]) successfully deleted
I20260808 23:30:31.149752 1641 ts_tablet_manager.cc:1918] T 7524509a469e48ac81f30d88f7cabdb5 P 27834cae70bf4f74a636e853ad33a807: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:30:31.149909 1641 log.cc:1196] T 7524509a469e48ac81f30d88f7cabdb5 P 27834cae70bf4f74a636e853ad33a807: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/wal/wals/7524509a469e48ac81f30d88f7cabdb5
I20260808 23:30:31.150470 1641 ts_tablet_manager.cc:1939] T 7524509a469e48ac81f30d88f7cabdb5 P 27834cae70bf4f74a636e853ad33a807: Deleting consensus metadata
I20260808 23:30:31.151889 914 catalog_manager.cc:4841] TS 27834cae70bf4f74a636e853ad33a807 (127.9.44.65:40463): tablet 7524509a469e48ac81f30d88f7cabdb5 (table test-table [id=b6980a27840545f682bf211a2dad7cfb]) successfully deleted
I20260808 23:30:31.155011 1603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:31.102233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45804 (local address 127.9.44.65:40463)
0808 23:30:31.102404 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:31.102411 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:31.124968 (+ 22557us) server_negotiation.cc:407] Connection header received
0808 23:30:31.125471 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:31.125477 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:31.125573 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:31.125662 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:31.131090 (+ 5428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:31.132280 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:31.138006 (+ 5726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:31.138627 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:31.138722 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:31.147903 (+ 9181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:31.147920 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:31.147941 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:31.147983 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:31.152076 (+ 4093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:31.152754 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:31.152764 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:31.152768 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:31.152942 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:31.153298 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:31.153303 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:31.153306 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:31.153785 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:31.154089 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:31.154405 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 23:30:31.154642 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 23:30:31.155789 1221 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:30:31.242137 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 1037
I20260808 23:30:31.267503 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 1235
Aug 08 23:30:31 dist-test-slave-6bbx krb5kdc[872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.66@KRBTEST.COM
I20260808 23:30:31.284472 1607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:31.268420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47512 (local address 127.9.44.66:38901)
0808 23:30:31.268626 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:31.268633 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:31.268680 (+ 47us) server_negotiation.cc:407] Connection header received
0808 23:30:31.268750 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:31.268756 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:31.268836 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:31.268955 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:31.270268 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:31.271455 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:31.272999 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:31.273676 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:31.273808 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:31.277841 (+ 4033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:31.277856 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:31.277879 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:31.277921 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:31.281537 (+ 3616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:31.282300 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:31.282310 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:31.282313 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:31.282501 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:31.282828 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:31.282834 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:31.282837 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:31.283272 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:31.283599 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:31.283829 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 23:30:31.284067 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 23:30:31.285300 1397 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:30:31.395792 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 1235
I20260808 23:30:31.417897 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 1409
Aug 08 23:30:31 dist-test-slave-6bbx krb5kdc[872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.67@KRBTEST.COM
I20260808 23:30:31.432925 1599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:31.418642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55974 (local address 127.9.44.67:37569)
0808 23:30:31.418823 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:31.418829 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:31.418849 (+ 20us) server_negotiation.cc:407] Connection header received
0808 23:30:31.418996 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:31.419001 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:31.419069 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:31.419163 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:31.420668 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:31.421384 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:31.422777 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:31.423170 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:31.423291 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:31.427553 (+ 4262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:31.427567 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:31.427583 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:31.427616 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:31.430246 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:31.430897 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:31.430904 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:31.430906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:31.431036 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:31.431389 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:31.431393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:31.431395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:31.431811 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:31.432099 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:31.432402 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 23:30:31.432626 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 23:30:31.433575 1575 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:30:31.519100 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 1409
I20260808 23:30:31.541402 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 890
I20260808 23:30:31.542492 986 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:30:31.649422 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 890
I20260808 23:30:51.677311 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:41945
--webserver_interface=127.9.44.126
--webserver_port=39343
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:41945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:30:52.007308 1650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:30:52.007599 1650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:30:52.007757 1650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:30:52.014714 1650 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 23:30:52.014806 1650 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 23:30:52.014861 1650 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 23:30:52.014931 1650 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:30:52.015007 1650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:30:52.015100 1650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:30:52.015164 1650 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:30:52.015237 1650 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:30:52.024616 1650 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:41945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:41945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=39343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:30:52.025735 1650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:30:52.026965 1650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:30:52.050869 1658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:52.050834 1659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:52.050904 1665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:30:52.051194 1650 server_base.cc:1034] running on GCE node
I20260808 23:30:52.052347 1650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:30:52.053753 1650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:30:52.054973 1650 hybrid_clock.cc:648] HybridClock initialized: now 1786231852054899 us; error 41 us; skew 500 ppm
I20260808 23:30:52.055315 1650 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:30:52 dist-test-slave-6bbx krb5kdc[872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231852, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:30:52.061965 1650 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:30:52.063553 1650 webserver.cc:460] Webserver started at http://127.9.44.126:39343/ using document root <none> and password file <none>
I20260808 23:30:52.064316 1650 fs_manager.cc:359] Metadata directory not provided
I20260808 23:30:52.064446 1650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:30:52.070066 1650 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260808 23:30:52.072880 1673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:52.074131 1650 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:30:52.074316 1650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "20626351a0a84f9688ac89ec3cd9d37b"
format_stamp: "Formatted at 2026-08-08 23:30:28 on dist-test-slave-6bbx"
server_key: "0974298a5dbd7cea4a3249795db8dfdc"
server_key_iv: "1ea1c7a961da6a5d1371eccfd8695962"
server_key_version: "encryptionkey@0"
I20260808 23:30:52.075367 1650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:30:52.102763 1650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:30:52.106525 1650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:30:52.106792 1650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:30:52.124429 1650 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:41945
I20260808 23:30:52.124469 1743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:41945 every 8 connection(s)
I20260808 23:30:52.125874 1650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:30:52.132668 1744 sys_catalog.cc:263] Verifying existing consensus state
I20260808 23:30:52.134598 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 1650
I20260808 23:30:52.135561 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:40463
--local_ip_for_outbound_sockets=127.9.44.65
--tserver_master_addrs=127.9.44.126:41945
--webserver_port=44233
--webserver_interface=127.9.44.65
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:30:52.136538 1744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b: Bootstrap starting.
I20260808 23:30:52.158433 1744 log.cc:824] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b: Log is configured to *not* fsync() on all Append() calls
I20260808 23:30:52.170184 1744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:30:52.170670 1744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b: Bootstrap complete.
I20260808 23:30:52.179513 1744 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20626351a0a84f9688ac89ec3cd9d37b" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 41945 } }
I20260808 23:30:52.180364 1744 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20626351a0a84f9688ac89ec3cd9d37b, State: Initialized, Role: FOLLOWER
I20260808 23:30:52.180917 1744 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20626351a0a84f9688ac89ec3cd9d37b" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 41945 } }
I20260808 23:30:52.181141 1744 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:30:52.181254 1744 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:30:52.181412 1744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [term 1 FOLLOWER]: Advancing to term 2
I20260808 23:30:52.185118 1744 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20626351a0a84f9688ac89ec3cd9d37b" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 41945 } }
I20260808 23:30:52.185604 1744 leader_election.cc:302] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20626351a0a84f9688ac89ec3cd9d37b; no voters:
I20260808 23:30:52.186204 1744 leader_election.cc:288] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 23:30:52.186347 1750 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [term 2 FOLLOWER]: Leader election won for term 2
I20260808 23:30:52.187947 1750 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [term 2 LEADER]: Becoming Leader. State: Replica: 20626351a0a84f9688ac89ec3cd9d37b, State: Running, Role: LEADER
I20260808 23:30:52.188505 1744 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:30:52.188458 1750 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20626351a0a84f9688ac89ec3cd9d37b" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 41945 } }
I20260808 23:30:52.190887 1754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20626351a0a84f9688ac89ec3cd9d37b. Latest consensus state: current_term: 2 leader_uuid: "20626351a0a84f9688ac89ec3cd9d37b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20626351a0a84f9688ac89ec3cd9d37b" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 41945 } } }
I20260808 23:30:52.191423 1754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [sys.catalog]: This master's current role is: LEADER
I20260808 23:30:52.199245 1759 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:30:52.199091 1752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "20626351a0a84f9688ac89ec3cd9d37b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20626351a0a84f9688ac89ec3cd9d37b" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 41945 } } }
I20260808 23:30:52.199466 1752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [sys.catalog]: This master's current role is: LEADER
I20260808 23:30:52.207756 1759 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:30:52.209236 1759 catalog_manager.cc:1259] Loaded cluster ID: f9646ad24674490abeab8d02016cc839
I20260808 23:30:52.209393 1759 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:30:52.213534 1759 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:30:52.215163 1759 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b: Loaded TSK: 0
I20260808 23:30:52.258003 1759 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b: Generated new TSK 1
I20260808 23:30:52.258396 1759 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 23:30:52.509930 1747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:30:52.510186 1747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:30:52.510295 1747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:30:52.517284 1747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:30:52.517385 1747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:30:52.517517 1747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:30:52.526757 1747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:40463
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=44233
--tserver_master_addrs=127.9.44.126:41945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:30:52.527899 1747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:30:52.529174 1747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:30:52.541435 1786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:52.541592 1784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:52.541687 1783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:30:52.542598 1747 server_base.cc:1034] running on GCE node
I20260808 23:30:52.543283 1747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:30:52.544421 1747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:30:52.545665 1747 hybrid_clock.cc:648] HybridClock initialized: now 1786231852545592 us; error 60 us; skew 500 ppm
I20260808 23:30:52.546023 1747 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:30:52 dist-test-slave-6bbx krb5kdc[872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231852, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:30:52.549841 1747 init.cc:377] Logged in from keytab as kudu/127.9.44.65@KRBTEST.COM (short username kudu)
I20260808 23:30:52.551460 1747 webserver.cc:460] Webserver started at http://127.9.44.65:44233/ using document root <none> and password file <none>
I20260808 23:30:52.552277 1747 fs_manager.cc:359] Metadata directory not provided
I20260808 23:30:52.552431 1747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:30:52.559208 1747 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 23:30:52.562407 1799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:52.563922 1747 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:30:52.564083 1747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "27834cae70bf4f74a636e853ad33a807"
format_stamp: "Formatted at 2026-08-08 23:30:29 on dist-test-slave-6bbx"
server_key: "706b1eb6906a4672ba3349a6a49d4432"
server_key_iv: "415485939e3d056a41c252a7ae6a8d50"
server_key_version: "encryptionkey@0"
I20260808 23:30:52.565331 1747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:30:52.601480 1747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:30:52.605495 1747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:30:52.605742 1747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:30:52.606663 1747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:30:52.607956 1747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:30:52.608047 1747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:52.608152 1747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:30:52.608224 1747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:52.636634 1747 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:40463
I20260808 23:30:52.636713 1955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:40463 every 8 connection(s)
I20260808 23:30:52.637948 1747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:30:52.647152 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 1747
I20260808 23:30:52.647919 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:38901
--local_ip_for_outbound_sockets=127.9.44.66
--tserver_master_addrs=127.9.44.126:41945
--webserver_port=45181
--webserver_interface=127.9.44.66
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:30:52 dist-test-slave-6bbx krb5kdc[872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231852, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:30:52.660777 1960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:52.640802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.65:59197)
0808 23:30:52.641609 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:52.641667 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:52.643340 (+ 1673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:52.644743 (+ 1403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:52.644750 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:52.645231 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:30:52.645758 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:52.645776 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:52.647777 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:52.647781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:52.649036 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:52.649050 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:52.650317 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:52.650322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:52.650506 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:52.651085 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:30:52.651106 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:30:52.653287 (+ 2181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:30:52.657039 (+ 3752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:52.657045 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:52.657055 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:52.657423 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:52.657862 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:52.657865 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:52.657867 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:52.658025 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:52.658789 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:30:52.658794 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:30:52.659050 (+ 256us) client_negotiation.cc:769] Sending connection context
0808 23:30:52.659255 (+ 205us) client_negotiation.cc:240] Negotiation successful
0808 23:30:52.659584 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":326,"threads_started":1}
I20260808 23:30:52.660792 1959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:52.640586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:59197 (local address 127.9.44.126:41945)
0808 23:30:52.641140 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:52.641152 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:52.641811 (+ 659us) server_negotiation.cc:407] Connection header received
0808 23:30:52.643930 (+ 2119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:52.643950 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:52.644276 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:52.644579 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:52.645919 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:52.647564 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:52.649195 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:52.650135 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:52.650309 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:52.653563 (+ 3254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:52.653572 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:52.653588 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:52.653628 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:52.656914 (+ 3286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:52.657550 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:52.657558 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:52.657565 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:52.657765 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:52.658123 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:52.658127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:52.658130 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:52.658678 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:52.658857 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:52.659490 (+ 633us) server_negotiation.cc:299] Negotiation successful
0808 23:30:52.659901 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":245,"threads_started":1}
I20260808 23:30:52.663328 1957 heartbeater.cc:347] Connected to a master server at 127.9.44.126:41945
I20260808 23:30:52.663573 1957 heartbeater.cc:464] Registering TS with master...
I20260808 23:30:52.664274 1957 heartbeater.cc:511] Master 127.9.44.126:41945 requested a full tablet report, sending...
I20260808 23:30:52.666760 1695 ts_manager.cc:194] Registered new tserver with Master: 27834cae70bf4f74a636e853ad33a807 (127.9.44.65:40463)
I20260808 23:30:52.668210 1695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.65@KRBTEST.COM'} at 127.9.44.65:59197
W20260808 23:30:52.974757 1961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:30:52.975031 1961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:30:52.975172 1961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:30:52.982308 1961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:30:52.982429 1961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:30:52.982584 1961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:30:52.991995 1961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:38901
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=45181
--tserver_master_addrs=127.9.44.126:41945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:30:52.993160 1961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:30:52.994403 1961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:30:53.006969 1971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:53.007037 1972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:53.007041 1974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:30:53.007758 1961 server_base.cc:1034] running on GCE node
I20260808 23:30:53.008375 1961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:30:53.009239 1961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:30:53.010439 1961 hybrid_clock.cc:648] HybridClock initialized: now 1786231853010400 us; error 42 us; skew 500 ppm
I20260808 23:30:53.010735 1961 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:30:53 dist-test-slave-6bbx krb5kdc[872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231853, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:30:53.013674 1961 init.cc:377] Logged in from keytab as kudu/127.9.44.66@KRBTEST.COM (short username kudu)
I20260808 23:30:53.015026 1961 webserver.cc:460] Webserver started at http://127.9.44.66:45181/ using document root <none> and password file <none>
I20260808 23:30:53.015647 1961 fs_manager.cc:359] Metadata directory not provided
I20260808 23:30:53.015829 1961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:30:53.020816 1961 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260808 23:30:53.023298 1985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:53.024513 1961 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 23:30:53.024670 1961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "db14e75d01c14755b969b34a0b3e4900"
format_stamp: "Formatted at 2026-08-08 23:30:30 on dist-test-slave-6bbx"
server_key: "1071a4017579a65def535a7cdc803bc6"
server_key_iv: "91d887173faaa0a4fdda8d5a7f11c016"
server_key_version: "encryptionkey@0"
I20260808 23:30:53.025580 1961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:30:53.042639 1961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:30:53.046216 1961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:30:53.046449 1961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:30:53.047317 1961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:30:53.048516 1961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:30:53.048612 1961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:53.048729 1961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:30:53.048799 1961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:53.084215 1961 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:38901
I20260808 23:30:53.084276 2138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:38901 every 8 connection(s)
I20260808 23:30:53.085551 1961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:30:53.088337 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 1961
I20260808 23:30:53.088960 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:37569
--local_ip_for_outbound_sockets=127.9.44.67
--tserver_master_addrs=127.9.44.126:41945
--webserver_port=42609
--webserver_interface=127.9.44.67
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 23:30:53 dist-test-slave-6bbx krb5kdc[872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231853, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:30:53.104496 1959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:53.087825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:39557 (local address 127.9.44.126:41945)
0808 23:30:53.088008 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:53.088016 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:53.089219 (+ 1203us) server_negotiation.cc:407] Connection header received
0808 23:30:53.091061 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:53.091066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:53.091146 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:53.091233 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:53.092726 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.093434 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.094933 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.095358 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.095470 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:53.098756 (+ 3286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:53.098761 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:53.098765 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:53.098799 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:53.101620 (+ 2821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:53.102339 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:53.102345 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:53.102348 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:53.102497 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:53.102875 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:53.102880 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:53.102882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:53.103206 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:53.103355 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:53.103982 (+ 627us) server_negotiation.cc:299] Negotiation successful
0808 23:30:53.104185 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 23:30:53.105157 2142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:53.088239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.66:39557)
0808 23:30:53.089011 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:53.089107 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:53.090800 (+ 1693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:53.091490 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:53.091498 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:53.092031 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:30:53.092555 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:53.092573 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.093596 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.093600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:53.094779 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:53.094802 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.095581 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.095588 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:53.095860 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:53.096488 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:30:53.096509 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:30:53.098542 (+ 2033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:30:53.101750 (+ 3208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:53.101756 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:53.101767 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:53.102183 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:53.102592 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:53.102595 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:53.102598 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:53.102767 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:53.103310 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:30:53.103315 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:30:53.103588 (+ 273us) client_negotiation.cc:769] Sending connection context
0808 23:30:53.103845 (+ 257us) client_negotiation.cc:240] Negotiation successful
0808 23:30:53.104183 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":309,"threads_started":1}
I20260808 23:30:53.106523 2140 heartbeater.cc:347] Connected to a master server at 127.9.44.126:41945
I20260808 23:30:53.106817 2140 heartbeater.cc:464] Registering TS with master...
I20260808 23:30:53.107626 2140 heartbeater.cc:511] Master 127.9.44.126:41945 requested a full tablet report, sending...
I20260808 23:30:53.109185 1695 ts_manager.cc:194] Registered new tserver with Master: db14e75d01c14755b969b34a0b3e4900 (127.9.44.66:38901)
I20260808 23:30:53.109949 1695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.66@KRBTEST.COM'} at 127.9.44.66:39557
W20260808 23:30:53.423705 2144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:30:53.423970 2144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:30:53.424101 2144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:30:53.431202 2144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:30:53.431295 2144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:30:53.431452 2144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:30:53.440670 2144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:37569
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=42609
--tserver_master_addrs=127.9.44.126:41945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:30:53.441773 2144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:30:53.443011 2144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:30:53.455556 2152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:53.455648 2154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:30:53.455642 2151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:30:53.456430 2144 server_base.cc:1034] running on GCE node
I20260808 23:30:53.456962 2144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:30:53.457767 2144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:30:53.458961 2144 hybrid_clock.cc:648] HybridClock initialized: now 1786231853458912 us; error 45 us; skew 500 ppm
I20260808 23:30:53.459240 2144 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:30:53 dist-test-slave-6bbx krb5kdc[872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231853, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:30:53.462296 2144 init.cc:377] Logged in from keytab as kudu/127.9.44.67@KRBTEST.COM (short username kudu)
I20260808 23:30:53.463626 2144 webserver.cc:460] Webserver started at http://127.9.44.67:42609/ using document root <none> and password file <none>
I20260808 23:30:53.464275 2144 fs_manager.cc:359] Metadata directory not provided
I20260808 23:30:53.464393 2144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:30:53.469483 2144 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 23:30:53.472165 2164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:53.473351 2144 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:30:53.473505 2144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "a951a5353a7e407981ce7ebc27ecabf4"
format_stamp: "Formatted at 2026-08-08 23:30:30 on dist-test-slave-6bbx"
server_key: "9e77796ec4a7a1180f15d98f6f1420df"
server_key_iv: "5c336e7edf9279c61ef79a4f89bb7b88"
server_key_version: "encryptionkey@0"
I20260808 23:30:53.474416 2144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:30:53.491919 2144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:30:53.495396 2144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:30:53.495642 2144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:30:53.496558 2144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:30:53.497813 2144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:30:53.497910 2144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:53.498032 2144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:30:53.498113 2144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:30:53.528381 2144 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:37569
I20260808 23:30:53.528496 2317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:37569 every 8 connection(s)
I20260808 23:30:53.529738 2144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:30:53.539976 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 2144
Aug 08 23:30:53 dist-test-slave-6bbx krb5kdc[872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231853, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:30:53.549031 1959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:53.532065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:33103 (local address 127.9.44.126:41945)
0808 23:30:53.532236 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:53.532241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:53.533306 (+ 1065us) server_negotiation.cc:407] Connection header received
0808 23:30:53.535083 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:53.535089 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:53.535167 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:53.535263 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:53.536742 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.537487 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.538989 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.539398 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.539510 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:53.543096 (+ 3586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:53.543102 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:53.543105 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:53.543136 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:53.545788 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:53.546573 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:53.546578 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:53.546582 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:53.546758 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:53.547309 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:53.547315 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:53.547318 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:53.547699 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:53.547872 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:53.548444 (+ 572us) server_negotiation.cc:299] Negotiation successful
0808 23:30:53.548661 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 23:30:53.549646 2320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:53.532485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.67:33103)
0808 23:30:53.533160 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:53.533195 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:53.534823 (+ 1628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:53.535520 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:53.535528 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:53.536059 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:30:53.536571 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:53.536588 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.537654 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.537658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:53.538845 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:53.538857 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.539616 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.539624 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:53.539900 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:53.540485 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:30:53.540518 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:30:53.542908 (+ 2390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:30:53.545921 (+ 3013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:53.545929 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:53.545944 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:53.546333 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:53.546898 (+ 565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:30:53.546904 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:30:53.546909 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:30:53.547173 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:30:53.547821 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:30:53.547826 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:30:53.548103 (+ 277us) client_negotiation.cc:769] Sending connection context
0808 23:30:53.548324 (+ 221us) client_negotiation.cc:240] Negotiation successful
0808 23:30:53.548659 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":271,"threads_started":1}
I20260808 23:30:53.550957 2318 heartbeater.cc:347] Connected to a master server at 127.9.44.126:41945
I20260808 23:30:53.551257 2318 heartbeater.cc:464] Registering TS with master...
I20260808 23:30:53.552158 2318 heartbeater.cc:511] Master 127.9.44.126:41945 requested a full tablet report, sending...
I20260808 23:30:53.553821 1695 ts_manager.cc:194] Registered new tserver with Master: a951a5353a7e407981ce7ebc27ecabf4 (127.9.44.67:37569)
I20260808 23:30:53.554873 1695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.67@KRBTEST.COM'} at 127.9.44.67:33103
I20260808 23:30:53.555166 2321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:53.540813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45908 (local address 127.9.44.126:41945)
0808 23:30:53.541296 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:53.541301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:53.541643 (+ 342us) server_negotiation.cc:407] Connection header received
0808 23:30:53.541829 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:53.541833 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:53.541891 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:53.541978 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:53.543582 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.544438 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.545809 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.546214 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.546323 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:53.549490 (+ 3167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:53.549495 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:53.549498 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:53.549526 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:53.552539 (+ 3013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:53.553227 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:53.553232 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:53.553235 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:53.553373 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:53.553805 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:53.553811 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:53.553813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:53.554077 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:53.554259 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:53.554632 (+ 373us) server_negotiation.cc:299] Negotiation successful
0808 23:30:53.554784 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":250,"threads_started":1}
I20260808 23:30:53.557588 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:30:53.563319 2321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:53.558437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45918 (local address 127.9.44.126:41945)
0808 23:30:53.558580 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:53.558584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:53.558952 (+ 368us) server_negotiation.cc:407] Connection header received
0808 23:30:53.559132 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:53.559136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:53.559259 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:53.559369 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:30:53.559778 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.560417 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.561745 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.562145 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.562230 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:53.562654 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:30:53.562904 (+ 250us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 23:30:53.563156 (+ 252us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45918: token expired
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 23:30:53.563407 2321 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45918: token expired
W20260808 23:30:53.564014 1586 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.9.44.126:41945)
I20260808 23:30:53.575526 2321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:53.564465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45932 (local address 127.9.44.126:41945)
0808 23:30:53.564605 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:53.564610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:53.564625 (+ 15us) server_negotiation.cc:407] Connection header received
0808 23:30:53.564781 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:53.564796 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:53.564846 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:53.564919 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:53.566214 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.566870 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.568195 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.568591 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.568707 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:53.571243 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:53.571248 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:53.571251 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:53.571280 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:53.573590 (+ 2310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:53.574104 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:53.574107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:53.574110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:53.574232 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:53.574498 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:53.574500 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:53.574502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:53.574765 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:53.574934 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:53.575140 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 23:30:53.575307 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 23:30:53.578722 1695 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45932:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:30:53.585945 1695 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:30:53.618110 2339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:53.598114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50208 (local address 127.9.44.65:40463)
0808 23:30:53.608042 (+ 9928us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:53.608054 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:53.610437 (+ 2383us) server_negotiation.cc:407] Connection header received
0808 23:30:53.610556 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:53.610564 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:53.610821 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:53.611014 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:30:53.611856 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.613481 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.615592 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.617009 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.617125 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:53.617323 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:53.617632 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 23:30:53.617801 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9747,"thread_start_us":10099,"threads_started":1}
I20260808 23:30:53.618255 2332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:53.597089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:40463 (local address 127.0.0.1:50208)
0808 23:30:53.610158 (+ 13069us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:53.610175 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:53.610304 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:53.611108 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:53.611115 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:53.611145 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:30:53.611635 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:53.611649 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.613612 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.613618 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:53.615434 (+ 1816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:53.615447 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.617226 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.617232 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:53.617413 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:53.617449 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 23:30:53.617558 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 23:30:53.617740 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12940,"mutex_wait_us":23,"thread_start_us":14160,"threads_started":1}
I20260808 23:30:53.619045 2337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:53.597019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:38901 (local address 127.0.0.1:60552)
0808 23:30:53.608285 (+ 11266us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:53.608334 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:53.608499 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:53.611954 (+ 3455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:53.611958 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:53.611982 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:30:53.612276 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:53.612285 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.614026 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.614031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:53.616224 (+ 2193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:53.616236 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.618264 (+ 2028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.618270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:53.618434 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:53.618467 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 23:30:53.618563 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 23:30:53.618685 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10523,"thread_start_us":13496,"threads_started":1}
I20260808 23:30:53.619076 2335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:53.597203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60552 (local address 127.9.44.66:38901)
0808 23:30:53.611054 (+ 13851us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:53.611062 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:53.611093 (+ 31us) server_negotiation.cc:407] Connection header received
0808 23:30:53.611188 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:53.611196 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:53.611764 (+ 568us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:53.611959 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:30:53.612422 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.613895 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.616400 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.617732 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.617842 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:53.617996 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:53.618638 (+ 642us) server_negotiation.cc:299] Negotiation successful
0808 23:30:53.618805 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13705,"thread_start_us":13600,"threads_started":1}
I20260808 23:30:53.619205 2331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:53.597599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:37569 (local address 127.0.0.1:33958)
0808 23:30:53.610450 (+ 12851us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:53.610465 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:53.610561 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:53.611439 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:53.611443 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:53.611472 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:30:53.611869 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:53.611882 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.613780 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.613784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:53.616046 (+ 2262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:53.616061 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.618198 (+ 2137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.618205 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:53.618636 (+ 431us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:53.618667 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 23:30:53.618730 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 23:30:53.618855 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12753,"mutex_wait_us":4,"thread_start_us":9793,"threads_started":1}
I20260808 23:30:53.619858 2341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:53.598792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33958 (local address 127.9.44.67:37569)
0808 23:30:53.608535 (+ 9743us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:53.608543 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:53.610669 (+ 2126us) server_negotiation.cc:407] Connection header received
0808 23:30:53.610763 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:53.610769 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:53.610977 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:53.611141 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:30:53.612097 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.613643 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.616237 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.617603 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.617727 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:53.617961 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:53.619330 (+ 1369us) server_negotiation.cc:299] Negotiation successful
0808 23:30:53.619490 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9573,"thread_start_us":10818,"threads_started":1}
I20260808 23:30:53.622248 1864 tablet_service.cc:1467] Processing CreateTablet for tablet 5c0ff80df4064f95a97c4c22dffa3028 (DEFAULT_TABLE table=test-table [id=57f04ad962e14d76bb2c13bb66fa7f3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:30:53.623435 1864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c0ff80df4064f95a97c4c22dffa3028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:30:53.622983 2227 tablet_service.cc:1467] Processing CreateTablet for tablet 5c0ff80df4064f95a97c4c22dffa3028 (DEFAULT_TABLE table=test-table [id=57f04ad962e14d76bb2c13bb66fa7f3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:30:53.622984 2047 tablet_service.cc:1467] Processing CreateTablet for tablet 5c0ff80df4064f95a97c4c22dffa3028 (DEFAULT_TABLE table=test-table [id=57f04ad962e14d76bb2c13bb66fa7f3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:30:53.624308 2227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c0ff80df4064f95a97c4c22dffa3028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:30:53.624336 2047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c0ff80df4064f95a97c4c22dffa3028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:30:53.635411 2342 tablet_bootstrap.cc:492] T 5c0ff80df4064f95a97c4c22dffa3028 P 27834cae70bf4f74a636e853ad33a807: Bootstrap starting.
I20260808 23:30:53.640184 2342 tablet_bootstrap.cc:654] T 5c0ff80df4064f95a97c4c22dffa3028 P 27834cae70bf4f74a636e853ad33a807: Neither blocks nor log segments found. Creating new log.
I20260808 23:30:53.641618 2342 log.cc:824] T 5c0ff80df4064f95a97c4c22dffa3028 P 27834cae70bf4f74a636e853ad33a807: Log is configured to *not* fsync() on all Append() calls
I20260808 23:30:53.642812 2344 tablet_bootstrap.cc:492] T 5c0ff80df4064f95a97c4c22dffa3028 P a951a5353a7e407981ce7ebc27ecabf4: Bootstrap starting.
I20260808 23:30:53.644182 2343 tablet_bootstrap.cc:492] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900: Bootstrap starting.
I20260808 23:30:53.648524 2344 tablet_bootstrap.cc:654] T 5c0ff80df4064f95a97c4c22dffa3028 P a951a5353a7e407981ce7ebc27ecabf4: Neither blocks nor log segments found. Creating new log.
I20260808 23:30:53.648542 2343 tablet_bootstrap.cc:654] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900: Neither blocks nor log segments found. Creating new log.
I20260808 23:30:53.649781 2343 log.cc:824] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900: Log is configured to *not* fsync() on all Append() calls
I20260808 23:30:53.649802 2344 log.cc:824] T 5c0ff80df4064f95a97c4c22dffa3028 P a951a5353a7e407981ce7ebc27ecabf4: Log is configured to *not* fsync() on all Append() calls
I20260808 23:30:53.652784 2343 tablet_bootstrap.cc:492] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900: No bootstrap required, opened a new log
I20260808 23:30:53.652941 2342 tablet_bootstrap.cc:492] T 5c0ff80df4064f95a97c4c22dffa3028 P 27834cae70bf4f74a636e853ad33a807: No bootstrap required, opened a new log
I20260808 23:30:53.653030 2343 ts_tablet_manager.cc:1397] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260808 23:30:53.653213 2342 ts_tablet_manager.cc:1397] T 5c0ff80df4064f95a97c4c22dffa3028 P 27834cae70bf4f74a636e853ad33a807: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.007s
I20260808 23:30:53.653299 2344 tablet_bootstrap.cc:492] T 5c0ff80df4064f95a97c4c22dffa3028 P a951a5353a7e407981ce7ebc27ecabf4: No bootstrap required, opened a new log
I20260808 23:30:53.653590 2344 ts_tablet_manager.cc:1397] T 5c0ff80df4064f95a97c4c22dffa3028 P a951a5353a7e407981ce7ebc27ecabf4: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260808 23:30:53.660193 2343 raft_consensus.cc:348] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db14e75d01c14755b969b34a0b3e4900" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38901 } } peers { permanent_uuid: "27834cae70bf4f74a636e853ad33a807" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40463 } } peers { permanent_uuid: "a951a5353a7e407981ce7ebc27ecabf4" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 37569 } }
I20260808 23:30:53.660531 2343 raft_consensus.cc:374] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:30:53.660633 2343 raft_consensus.cc:729] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db14e75d01c14755b969b34a0b3e4900, State: Initialized, Role: FOLLOWER
I20260808 23:30:53.661175 2343 consensus_queue.cc:260] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db14e75d01c14755b969b34a0b3e4900" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38901 } } peers { permanent_uuid: "27834cae70bf4f74a636e853ad33a807" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40463 } } peers { permanent_uuid: "a951a5353a7e407981ce7ebc27ecabf4" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 37569 } }
I20260808 23:30:53.662228 2140 heartbeater.cc:503] Master 127.9.44.126:41945 was elected leader, sending a full tablet report...
I20260808 23:30:53.662844 2343 ts_tablet_manager.cc:1428] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260808 23:30:53.663376 2342 raft_consensus.cc:348] T 5c0ff80df4064f95a97c4c22dffa3028 P 27834cae70bf4f74a636e853ad33a807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db14e75d01c14755b969b34a0b3e4900" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38901 } } peers { permanent_uuid: "27834cae70bf4f74a636e853ad33a807" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40463 } } peers { permanent_uuid: "a951a5353a7e407981ce7ebc27ecabf4" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 37569 } }
I20260808 23:30:53.663859 2342 raft_consensus.cc:374] T 5c0ff80df4064f95a97c4c22dffa3028 P 27834cae70bf4f74a636e853ad33a807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:30:53.663596 2344 raft_consensus.cc:348] T 5c0ff80df4064f95a97c4c22dffa3028 P a951a5353a7e407981ce7ebc27ecabf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db14e75d01c14755b969b34a0b3e4900" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38901 } } peers { permanent_uuid: "27834cae70bf4f74a636e853ad33a807" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40463 } } peers { permanent_uuid: "a951a5353a7e407981ce7ebc27ecabf4" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 37569 } }
I20260808 23:30:53.663970 2342 raft_consensus.cc:729] T 5c0ff80df4064f95a97c4c22dffa3028 P 27834cae70bf4f74a636e853ad33a807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27834cae70bf4f74a636e853ad33a807, State: Initialized, Role: FOLLOWER
I20260808 23:30:53.664003 2344 raft_consensus.cc:374] T 5c0ff80df4064f95a97c4c22dffa3028 P a951a5353a7e407981ce7ebc27ecabf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:30:53.664089 2344 raft_consensus.cc:729] T 5c0ff80df4064f95a97c4c22dffa3028 P a951a5353a7e407981ce7ebc27ecabf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a951a5353a7e407981ce7ebc27ecabf4, State: Initialized, Role: FOLLOWER
I20260808 23:30:53.664637 2344 consensus_queue.cc:260] T 5c0ff80df4064f95a97c4c22dffa3028 P a951a5353a7e407981ce7ebc27ecabf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db14e75d01c14755b969b34a0b3e4900" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38901 } } peers { permanent_uuid: "27834cae70bf4f74a636e853ad33a807" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40463 } } peers { permanent_uuid: "a951a5353a7e407981ce7ebc27ecabf4" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 37569 } }
I20260808 23:30:53.664642 2342 consensus_queue.cc:260] T 5c0ff80df4064f95a97c4c22dffa3028 P 27834cae70bf4f74a636e853ad33a807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db14e75d01c14755b969b34a0b3e4900" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38901 } } peers { permanent_uuid: "27834cae70bf4f74a636e853ad33a807" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40463 } } peers { permanent_uuid: "a951a5353a7e407981ce7ebc27ecabf4" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 37569 } }
I20260808 23:30:53.665693 2318 heartbeater.cc:503] Master 127.9.44.126:41945 was elected leader, sending a full tablet report...
I20260808 23:30:53.666405 2344 ts_tablet_manager.cc:1428] T 5c0ff80df4064f95a97c4c22dffa3028 P a951a5353a7e407981ce7ebc27ecabf4: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260808 23:30:53.667341 1957 heartbeater.cc:503] Master 127.9.44.126:41945 was elected leader, sending a full tablet report...
I20260808 23:30:53.667903 2342 ts_tablet_manager.cc:1428] T 5c0ff80df4064f95a97c4c22dffa3028 P 27834cae70bf4f74a636e853ad33a807: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260808 23:30:53.743705 2350 raft_consensus.cc:482] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:30:53.744055 2350 raft_consensus.cc:504] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db14e75d01c14755b969b34a0b3e4900" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38901 } } peers { permanent_uuid: "27834cae70bf4f74a636e853ad33a807" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40463 } } peers { permanent_uuid: "a951a5353a7e407981ce7ebc27ecabf4" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 37569 } }
I20260808 23:30:53.746093 2350 leader_election.cc:288] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27834cae70bf4f74a636e853ad33a807 (127.9.44.65:40463), a951a5353a7e407981ce7ebc27ecabf4 (127.9.44.67:37569)
I20260808 23:30:53.752480 2341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:53.746809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:52005 (local address 127.9.44.67:37569)
0808 23:30:53.747022 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:53.747028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:53.747340 (+ 312us) server_negotiation.cc:407] Connection header received
0808 23:30:53.747503 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:53.747508 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:53.747569 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:53.747645 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:30:53.748338 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.749027 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.750766 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.751536 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.751647 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:53.751773 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:53.752113 (+ 340us) server_negotiation.cc:299] Negotiation successful
0808 23:30:53.752239 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 23:30:53.752590 2356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:53.746390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:37569 (local address 127.9.44.66:52005)
0808 23:30:53.747239 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:53.747258 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:53.747376 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:53.747696 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:53.747700 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:53.747741 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:30:53.748182 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:53.748192 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.749160 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.749169 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:53.750626 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:53.750638 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.751815 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.751823 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:53.751959 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:53.751987 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 23:30:53.752069 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 23:30:53.752190 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"thread_start_us":527,"threads_started":1}
I20260808 23:30:53.752647 2353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:53.746274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:40463 (local address 127.9.44.66:60509)
0808 23:30:53.746923 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0808 23:30:53.746944 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 23:30:53.747292 (+ 348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:30:53.747683 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:30:53.747688 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:30:53.747716 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:30:53.748082 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:53.748092 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.749071 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.749076 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:53.750595 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:30:53.750608 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.751832 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.751839 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:30:53.751977 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:53.752000 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 23:30:53.752081 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 23:30:53.752197 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":20,"thread_start_us":317,"threads_started":1}
I20260808 23:30:53.752499 2339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:53.746537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:60509 (local address 127.9.44.65:40463)
0808 23:30:53.746737 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:53.746743 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:53.747044 (+ 301us) server_negotiation.cc:407] Connection header received
0808 23:30:53.747465 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:53.747470 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:53.747533 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:53.747608 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:30:53.748229 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.748932 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.750759 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.751543 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.751684 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:53.751776 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:53.752139 (+ 363us) server_negotiation.cc:299] Negotiation successful
0808 23:30:53.752258 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 23:30:53.753769 2259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c0ff80df4064f95a97c4c22dffa3028" candidate_uuid: "db14e75d01c14755b969b34a0b3e4900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a951a5353a7e407981ce7ebc27ecabf4" is_pre_election: true
I20260808 23:30:53.753863 1893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c0ff80df4064f95a97c4c22dffa3028" candidate_uuid: "db14e75d01c14755b969b34a0b3e4900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27834cae70bf4f74a636e853ad33a807" is_pre_election: true
I20260808 23:30:53.754388 2259 raft_consensus.cc:2454] T 5c0ff80df4064f95a97c4c22dffa3028 P a951a5353a7e407981ce7ebc27ecabf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db14e75d01c14755b969b34a0b3e4900 in term 0.
I20260808 23:30:53.754379 1893 raft_consensus.cc:2454] T 5c0ff80df4064f95a97c4c22dffa3028 P 27834cae70bf4f74a636e853ad33a807 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db14e75d01c14755b969b34a0b3e4900 in term 0.
I20260808 23:30:53.755067 1988 leader_election.cc:302] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27834cae70bf4f74a636e853ad33a807, db14e75d01c14755b969b34a0b3e4900; no voters:
I20260808 23:30:53.755415 2350 raft_consensus.cc:2781] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:30:53.755569 2350 raft_consensus.cc:482] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:30:53.755710 2350 raft_consensus.cc:3037] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:30:53.757871 2350 raft_consensus.cc:504] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db14e75d01c14755b969b34a0b3e4900" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38901 } } peers { permanent_uuid: "27834cae70bf4f74a636e853ad33a807" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40463 } } peers { permanent_uuid: "a951a5353a7e407981ce7ebc27ecabf4" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 37569 } }
I20260808 23:30:53.758570 2350 leader_election.cc:288] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900 [CANDIDATE]: Term 1 election: Requested vote from peers 27834cae70bf4f74a636e853ad33a807 (127.9.44.65:40463), a951a5353a7e407981ce7ebc27ecabf4 (127.9.44.67:37569)
I20260808 23:30:53.758937 1893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c0ff80df4064f95a97c4c22dffa3028" candidate_uuid: "db14e75d01c14755b969b34a0b3e4900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27834cae70bf4f74a636e853ad33a807"
I20260808 23:30:53.759156 1893 raft_consensus.cc:3037] T 5c0ff80df4064f95a97c4c22dffa3028 P 27834cae70bf4f74a636e853ad33a807 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:30:53.759156 2259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c0ff80df4064f95a97c4c22dffa3028" candidate_uuid: "db14e75d01c14755b969b34a0b3e4900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a951a5353a7e407981ce7ebc27ecabf4"
I20260808 23:30:53.759346 2259 raft_consensus.cc:3037] T 5c0ff80df4064f95a97c4c22dffa3028 P a951a5353a7e407981ce7ebc27ecabf4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:30:53.762444 1893 raft_consensus.cc:2454] T 5c0ff80df4064f95a97c4c22dffa3028 P 27834cae70bf4f74a636e853ad33a807 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db14e75d01c14755b969b34a0b3e4900 in term 1.
I20260808 23:30:53.762629 2259 raft_consensus.cc:2454] T 5c0ff80df4064f95a97c4c22dffa3028 P a951a5353a7e407981ce7ebc27ecabf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db14e75d01c14755b969b34a0b3e4900 in term 1.
I20260808 23:30:53.762876 1988 leader_election.cc:302] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27834cae70bf4f74a636e853ad33a807, db14e75d01c14755b969b34a0b3e4900; no voters:
I20260808 23:30:53.763136 2350 raft_consensus.cc:2781] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:30:53.763551 2350 raft_consensus.cc:686] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900 [term 1 LEADER]: Becoming Leader. State: Replica: db14e75d01c14755b969b34a0b3e4900, State: Running, Role: LEADER
I20260808 23:30:53.764008 2350 consensus_queue.cc:237] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db14e75d01c14755b969b34a0b3e4900" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38901 } } peers { permanent_uuid: "27834cae70bf4f74a636e853ad33a807" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40463 } } peers { permanent_uuid: "a951a5353a7e407981ce7ebc27ecabf4" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 37569 } }
I20260808 23:30:53.768181 1694 catalog_manager.cc:5495] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900 reported cstate change: term changed from 0 to 1, leader changed from <none> to db14e75d01c14755b969b34a0b3e4900 (127.9.44.66). New cstate: current_term: 1 leader_uuid: "db14e75d01c14755b969b34a0b3e4900" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db14e75d01c14755b969b34a0b3e4900" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27834cae70bf4f74a636e853ad33a807" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a951a5353a7e407981ce7ebc27ecabf4" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 37569 } health_report { overall_health: UNKNOWN } } }
W20260808 23:30:53.781162 2319 tablet.cc:2369] T 5c0ff80df4064f95a97c4c22dffa3028 P a951a5353a7e407981ce7ebc27ecabf4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:30:53.805922 2335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:53.801240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60554 (local address 127.9.44.66:38901)
0808 23:30:53.801413 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:53.801420 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:53.801439 (+ 19us) server_negotiation.cc:407] Connection header received
0808 23:30:53.801610 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:53.801614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:53.801684 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:53.801813 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:30:53.802096 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.802801 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.804163 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.804557 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.804645 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:53.805044 (+ 399us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:30:53.805472 (+ 428us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:30:53.805571 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:53.805646 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 23:30:53.805714 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 23:30:53.813326 2259 raft_consensus.cc:1261] T 5c0ff80df4064f95a97c4c22dffa3028 P a951a5353a7e407981ce7ebc27ecabf4 [term 1 FOLLOWER]: Refusing update from remote peer db14e75d01c14755b969b34a0b3e4900: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:30:53.813417 1893 raft_consensus.cc:1261] T 5c0ff80df4064f95a97c4c22dffa3028 P 27834cae70bf4f74a636e853ad33a807 [term 1 FOLLOWER]: Refusing update from remote peer db14e75d01c14755b969b34a0b3e4900: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:30:53.814229 2350 consensus_queue.cc:1035] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a951a5353a7e407981ce7ebc27ecabf4" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 37569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:30:53.814543 2357 consensus_queue.cc:1035] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27834cae70bf4f74a636e853ad33a807" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:30:53.826336 2360 mvcc.cc:205] Tried to move back new op lower bound from 7316405673210892288 to 7316405673021034496. Current Snapshot: MvccSnapshot[applied={T|T < 7316405673210892288}]
I20260808 23:30:53.837330 1695 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:30:53.837568 1695 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:30:53.840874 1695 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b: Sending DeleteTablet for 3 replicas of tablet 5c0ff80df4064f95a97c4c22dffa3028
I20260808 23:30:53.842027 1864 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c0ff80df4064f95a97c4c22dffa3028 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:30:53 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:50208
I20260808 23:30:53.842006 2047 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c0ff80df4064f95a97c4c22dffa3028 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:30:53 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:60552
I20260808 23:30:53.842698 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 1747
I20260808 23:30:53.842718 2227 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c0ff80df4064f95a97c4c22dffa3028 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:30:53 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:33958
I20260808 23:30:53.844336 2374 tablet_replica.cc:330] T 5c0ff80df4064f95a97c4c22dffa3028 P a951a5353a7e407981ce7ebc27ecabf4: stopping tablet replica
I20260808 23:30:53.844729 2374 raft_consensus.cc:2229] T 5c0ff80df4064f95a97c4c22dffa3028 P a951a5353a7e407981ce7ebc27ecabf4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:30:53.845196 2374 raft_consensus.cc:2258] T 5c0ff80df4064f95a97c4c22dffa3028 P a951a5353a7e407981ce7ebc27ecabf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:30:53.845870 2372 tablet_replica.cc:330] T 5c0ff80df4064f95a97c4c22dffa3028 P 27834cae70bf4f74a636e853ad33a807: stopping tablet replica
I20260808 23:30:53.846248 2372 raft_consensus.cc:2229] T 5c0ff80df4064f95a97c4c22dffa3028 P 27834cae70bf4f74a636e853ad33a807 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:30:53.846686 2372 raft_consensus.cc:2258] T 5c0ff80df4064f95a97c4c22dffa3028 P 27834cae70bf4f74a636e853ad33a807 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:30:53.847451 2373 tablet_replica.cc:330] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900: stopping tablet replica
I20260808 23:30:53.847838 2373 raft_consensus.cc:2229] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:30:53.848527 2373 raft_consensus.cc:2258] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:30:53.849190 2372 ts_tablet_manager.cc:1905] T 5c0ff80df4064f95a97c4c22dffa3028 P 27834cae70bf4f74a636e853ad33a807: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:30:53.850363 2373 ts_tablet_manager.cc:1905] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:30:53.856056 2372 ts_tablet_manager.cc:1918] T 5c0ff80df4064f95a97c4c22dffa3028 P 27834cae70bf4f74a636e853ad33a807: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:30:53.856222 2372 log.cc:1196] T 5c0ff80df4064f95a97c4c22dffa3028 P 27834cae70bf4f74a636e853ad33a807: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/wal/wals/5c0ff80df4064f95a97c4c22dffa3028
I20260808 23:30:53.856787 2372 ts_tablet_manager.cc:1939] T 5c0ff80df4064f95a97c4c22dffa3028 P 27834cae70bf4f74a636e853ad33a807: Deleting consensus metadata
I20260808 23:30:53.857733 2373 ts_tablet_manager.cc:1918] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:30:53.857985 2373 log.cc:1196] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/5c0ff80df4064f95a97c4c22dffa3028
I20260808 23:30:53.858543 1674 catalog_manager.cc:4841] TS 27834cae70bf4f74a636e853ad33a807 (127.9.44.65:40463): tablet 5c0ff80df4064f95a97c4c22dffa3028 (table test-table [id=57f04ad962e14d76bb2c13bb66fa7f3c]) successfully deleted
I20260808 23:30:53.858613 2373 ts_tablet_manager.cc:1939] T 5c0ff80df4064f95a97c4c22dffa3028 P db14e75d01c14755b969b34a0b3e4900: Deleting consensus metadata
I20260808 23:30:53.859009 2374 ts_tablet_manager.cc:1905] T 5c0ff80df4064f95a97c4c22dffa3028 P a951a5353a7e407981ce7ebc27ecabf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:30:53.860394 1675 catalog_manager.cc:4841] TS db14e75d01c14755b969b34a0b3e4900 (127.9.44.66:38901): tablet 5c0ff80df4064f95a97c4c22dffa3028 (table test-table [id=57f04ad962e14d76bb2c13bb66fa7f3c]) successfully deleted
I20260808 23:30:53.861882 2339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:53.843796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50222 (local address 127.9.44.65:40463)
0808 23:30:53.843968 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:53.843975 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:53.843996 (+ 21us) server_negotiation.cc:407] Connection header received
0808 23:30:53.844053 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:53.844058 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:53.844157 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:53.844273 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:53.846657 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.847979 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.850340 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:53.850953 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:53.851075 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:53.854533 (+ 3458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:53.854547 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:53.854562 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:53.854599 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:53.858581 (+ 3982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:53.859497 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:53.859506 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:53.859509 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:53.859708 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:53.860140 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:53.860145 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:53.860148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:53.860601 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:53.860860 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:53.861332 (+ 472us) server_negotiation.cc:299] Negotiation successful
0808 23:30:53.861545 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 23:30:53.862932 1948 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:30:53.865597 2374 ts_tablet_manager.cc:1918] T 5c0ff80df4064f95a97c4c22dffa3028 P a951a5353a7e407981ce7ebc27ecabf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:30:53.865763 2374 log.cc:1196] T 5c0ff80df4064f95a97c4c22dffa3028 P a951a5353a7e407981ce7ebc27ecabf4: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/5c0ff80df4064f95a97c4c22dffa3028
I20260808 23:30:53.866266 2374 ts_tablet_manager.cc:1939] T 5c0ff80df4064f95a97c4c22dffa3028 P a951a5353a7e407981ce7ebc27ecabf4: Deleting consensus metadata
I20260808 23:30:53.867748 1677 catalog_manager.cc:4841] TS a951a5353a7e407981ce7ebc27ecabf4 (127.9.44.67:37569): tablet 5c0ff80df4064f95a97c4c22dffa3028 (table test-table [id=57f04ad962e14d76bb2c13bb66fa7f3c]) successfully deleted
I20260808 23:30:53.980747 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 1747
I20260808 23:30:54.005834 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 1961
I20260808 23:30:54.023455 2335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:54.006661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60568 (local address 127.9.44.66:38901)
0808 23:30:54.006874 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:54.006880 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:54.006899 (+ 19us) server_negotiation.cc:407] Connection header received
0808 23:30:54.007097 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:54.007101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:54.007176 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:54.007267 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:54.008675 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:54.009535 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:54.010817 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:54.011275 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:54.011386 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:54.017664 (+ 6278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:54.017677 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:54.017692 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:54.017743 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:54.020694 (+ 2951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:54.021340 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:54.021347 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:54.021349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:54.021516 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:54.021893 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:54.021897 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:54.021899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:54.022296 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:54.022614 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:54.022902 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 23:30:54.023116 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 23:30:54.024216 2129 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:30:54.133534 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 1961
I20260808 23:30:54.158429 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 2144
I20260808 23:30:54.171501 2341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:30:54.159170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33970 (local address 127.9.44.67:37569)
0808 23:30:54.159365 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 23:30:54.159371 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:30:54.159392 (+ 21us) server_negotiation.cc:407] Connection header received
0808 23:30:54.159553 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:30:54.159557 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:30:54.159624 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:30:54.159727 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:30:54.161026 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:54.161751 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:54.163140 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:30:54.163540 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:30:54.163651 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:30:54.166152 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:30:54.166166 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:30:54.166181 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:30:54.166217 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:30:54.168829 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:54.169511 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:54.169519 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:54.169521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:54.169664 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:30:54.170033 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:30:54.170036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:30:54.170038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:30:54.170391 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:30:54.170694 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0808 23:30:54.170996 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 23:30:54.171203 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 23:30:54.172372 2312 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:30:54.258498 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 2144
I20260808 23:30:54.279930 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 1650
I20260808 23:30:54.281068 1738 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:30:54.390874 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 1650
I20260808 23:31:14.419523 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:41945
--webserver_interface=127.9.44.126
--webserver_port=39343
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:41945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:31:14.749482 2383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:14.749750 2383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:14.749879 2383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:14.756809 2383 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 23:31:14.756891 2383 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 23:31:14.756949 2383 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 23:31:14.757016 2383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:31:14.757086 2383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:14.757186 2383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:14.757241 2383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:31:14.757316 2383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:31:14.766595 2383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:41945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:41945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=39343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:14.767704 2383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:14.768923 2383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:14.780045 2391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:14.780113 2398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:14.780083 2395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:14.780347 2383 server_base.cc:1034] running on GCE node
I20260808 23:31:14.781334 2383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:14.782584 2383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:14.783782 2383 hybrid_clock.cc:648] HybridClock initialized: now 1786231874783717 us; error 38 us; skew 500 ppm
I20260808 23:31:14.784078 2383 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:31:14 dist-test-slave-6bbx krb5kdc[872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231874, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:31:14.786998 2383 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:31:14.788332 2383 webserver.cc:460] Webserver started at http://127.9.44.126:39343/ using document root <none> and password file <none>
I20260808 23:31:14.788971 2383 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:14.789070 2383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:14.794126 2383 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 23:31:14.796738 2409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:14.797969 2383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:31:14.798122 2383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "20626351a0a84f9688ac89ec3cd9d37b"
format_stamp: "Formatted at 2026-08-08 23:30:28 on dist-test-slave-6bbx"
server_key: "0974298a5dbd7cea4a3249795db8dfdc"
server_key_iv: "1ea1c7a961da6a5d1371eccfd8695962"
server_key_version: "encryptionkey@0"
I20260808 23:31:14.799027 2383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:14.815488 2383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:14.818962 2383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:14.819203 2383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:14.836212 2383 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:41945
I20260808 23:31:14.836269 2473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:41945 every 8 connection(s)
I20260808 23:31:14.837599 2383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:31:14.844218 2474 sys_catalog.cc:263] Verifying existing consensus state
I20260808 23:31:14.847083 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 2383
I20260808 23:31:14.847334 2474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b: Bootstrap starting.
I20260808 23:31:14.847965 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:40463
--local_ip_for_outbound_sockets=127.9.44.65
--tserver_master_addrs=127.9.44.126:41945
--webserver_port=44233
--webserver_interface=127.9.44.65
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:31:14.866779 2474 log.cc:824] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:14.882544 2474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 23:31:14.883010 2474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b: Bootstrap complete.
I20260808 23:31:14.891593 2474 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20626351a0a84f9688ac89ec3cd9d37b" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 41945 } }
I20260808 23:31:14.892498 2474 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20626351a0a84f9688ac89ec3cd9d37b, State: Initialized, Role: FOLLOWER
I20260808 23:31:14.893049 2474 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20626351a0a84f9688ac89ec3cd9d37b" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 41945 } }
I20260808 23:31:14.893265 2474 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:31:14.893390 2474 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:31:14.893551 2474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [term 2 FOLLOWER]: Advancing to term 3
I20260808 23:31:14.897197 2474 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20626351a0a84f9688ac89ec3cd9d37b" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 41945 } }
I20260808 23:31:14.897660 2474 leader_election.cc:302] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20626351a0a84f9688ac89ec3cd9d37b; no voters:
I20260808 23:31:14.898263 2474 leader_election.cc:288] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 23:31:14.898402 2483 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [term 3 FOLLOWER]: Leader election won for term 3
I20260808 23:31:14.899477 2483 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [term 3 LEADER]: Becoming Leader. State: Replica: 20626351a0a84f9688ac89ec3cd9d37b, State: Running, Role: LEADER
I20260808 23:31:14.900194 2483 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20626351a0a84f9688ac89ec3cd9d37b" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 41945 } }
I20260808 23:31:14.900396 2474 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:31:14.902876 2485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20626351a0a84f9688ac89ec3cd9d37b. Latest consensus state: current_term: 3 leader_uuid: "20626351a0a84f9688ac89ec3cd9d37b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20626351a0a84f9688ac89ec3cd9d37b" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 41945 } } }
I20260808 23:31:14.903294 2485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [sys.catalog]: This master's current role is: LEADER
I20260808 23:31:14.903364 2484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "20626351a0a84f9688ac89ec3cd9d37b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20626351a0a84f9688ac89ec3cd9d37b" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 41945 } } }
I20260808 23:31:14.903735 2484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b [sys.catalog]: This master's current role is: LEADER
I20260808 23:31:14.914255 2493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:31:14.922484 2493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:31:14.923923 2493 catalog_manager.cc:1259] Loaded cluster ID: f9646ad24674490abeab8d02016cc839
I20260808 23:31:14.924072 2493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:31:14.928056 2493 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:31:14.929690 2493 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b: Loaded TSK: 0
I20260808 23:31:14.929893 2493 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 20626351a0a84f9688ac89ec3cd9d37b: Loaded TSK: 1
F20260808 23:31:14.931059 2493 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f3feba8f9ad google::LogMessage::Fail() at ??:0
@ 0x7f3feba9288a google::LogMessage::SendToLog() at ??:0
@ 0x7f3feba8f4de google::LogMessage::Flush() at ??:0
@ 0x7f3feba90a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f3fef7f37d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f3fef7e273e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f40014be1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f40014bd783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f40015492a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f40015490c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f40015935e9 std::function<>::operator()() at ??:0
@ 0x7f40014c1cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f40014bfc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f4001547610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f4001547439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3fee6db345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3fee6de91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f3fee6de799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f3fee6b9eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3ff19936db start_thread at ??:0
@ 0x7f3fea92961f clone at ??:0
W20260808 23:31:15.240985 2476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:15.241266 2476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:15.241386 2476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:15.248574 2476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:15.248675 2476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:15.248826 2476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:31:15.258402 2476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:40463
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=44233
--tserver_master_addrs=127.9.44.126:41945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:15.259584 2476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:15.260872 2476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:15.273258 2543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:15.273344 2548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:15.273506 2540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:15.274276 2476 server_base.cc:1034] running on GCE node
I20260808 23:31:15.274850 2476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:15.275735 2476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:15.276932 2476 hybrid_clock.cc:648] HybridClock initialized: now 1786231875276889 us; error 43 us; skew 500 ppm
I20260808 23:31:15.277220 2476 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:31:15 dist-test-slave-6bbx krb5kdc[872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231875, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:31:15.280133 2476 init.cc:377] Logged in from keytab as kudu/127.9.44.65@KRBTEST.COM (short username kudu)
I20260808 23:31:15.281401 2476 webserver.cc:460] Webserver started at http://127.9.44.65:44233/ using document root <none> and password file <none>
I20260808 23:31:15.282042 2476 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:15.282178 2476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:15.287117 2476 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:31:15.289721 2557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:15.290908 2476 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 23:31:15.291066 2476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "27834cae70bf4f74a636e853ad33a807"
format_stamp: "Formatted at 2026-08-08 23:30:29 on dist-test-slave-6bbx"
server_key: "706b1eb6906a4672ba3349a6a49d4432"
server_key_iv: "415485939e3d056a41c252a7ae6a8d50"
server_key_version: "encryptionkey@0"
I20260808 23:31:15.292034 2476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:15.311311 2476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:15.314807 2476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:15.315037 2476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:15.315960 2476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:31:15.317214 2476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:31:15.317310 2476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:15.317440 2476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:31:15.317512 2476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:15.347417 2476 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:40463
I20260808 23:31:15.347468 2708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:40463 every 8 connection(s)
I20260808 23:31:15.348805 2476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:31:15.354698 2714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:15.351558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.65:48965)
0808 23:31:15.353655 (+ 2097us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:15.353888 (+ 233us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1842,"thread_start_us":1621,"threads_started":1}
W20260808 23:31:15.355298 2710 heartbeater.cc:644] Failed to heartbeat to 127.9.44.126:41945 (0 consecutive failures): Network error: Failed to ping master at 127.9.44.126:41945: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
I20260808 23:31:15.356020 2714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:15.355738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.65:53071)
0808 23:31:15.355857 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:15.355921 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 23:31:15.356750 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 2476
I20260808 23:31:15.356812 2714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:15.356529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.65:34849)
0808 23:31:15.356633 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:15.356699 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 23:31:15.357024 2710 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 23:31:15.357378 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:38901
--local_ip_for_outbound_sockets=127.9.44.66
--tserver_master_addrs=127.9.44.126:41945
--webserver_port=45181
--webserver_interface=127.9.44.66
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:31:15.679466 2715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:15.679740 2715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:15.679881 2715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:15.686774 2715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:15.686869 2715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:15.687013 2715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:31:15.696146 2715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:38901
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=45181
--tserver_master_addrs=127.9.44.126:41945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:15.697266 2715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:15.698446 2715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:15.710405 2722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:15.710405 2725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:15.710513 2723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:15.711474 2715 server_base.cc:1034] running on GCE node
I20260808 23:31:15.712090 2715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:15.712958 2715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:15.714157 2715 hybrid_clock.cc:648] HybridClock initialized: now 1786231875714101 us; error 45 us; skew 500 ppm
I20260808 23:31:15.714452 2715 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:31:15 dist-test-slave-6bbx krb5kdc[872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231875, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:31:15.717386 2715 init.cc:377] Logged in from keytab as kudu/127.9.44.66@KRBTEST.COM (short username kudu)
I20260808 23:31:15.718771 2715 webserver.cc:460] Webserver started at http://127.9.44.66:45181/ using document root <none> and password file <none>
I20260808 23:31:15.719399 2715 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:15.719524 2715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:15.724594 2715 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 23:31:15.727126 2735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:15.728364 2715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 23:31:15.728523 2715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "db14e75d01c14755b969b34a0b3e4900"
format_stamp: "Formatted at 2026-08-08 23:30:30 on dist-test-slave-6bbx"
server_key: "1071a4017579a65def535a7cdc803bc6"
server_key_iv: "91d887173faaa0a4fdda8d5a7f11c016"
server_key_version: "encryptionkey@0"
I20260808 23:31:15.729449 2715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:15.752496 2715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:15.756059 2715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:15.756299 2715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:15.757184 2715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:31:15.758461 2715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:31:15.758567 2715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:15.758671 2715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:31:15.758741 2715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:15.788393 2715 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:38901
I20260808 23:31:15.788455 2893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:38901 every 8 connection(s)
I20260808 23:31:15.789721 2715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:31:15.793723 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 2715
I20260808 23:31:15.794193 2899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:15.792401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.66:44327)
0808 23:31:15.793082 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:15.793324 (+ 242us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":255,"threads_started":1}
I20260808 23:31:15.794286 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:37569
--local_ip_for_outbound_sockets=127.9.44.67
--tserver_master_addrs=127.9.44.126:41945
--webserver_port=42609
--webserver_interface=127.9.44.67
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:31:15.794906 2894 heartbeater.cc:644] Failed to heartbeat to 127.9.44.126:41945 (0 consecutive failures): Network error: Failed to ping master at 127.9.44.126:41945: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
I20260808 23:31:15.795711 2899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:15.795367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.66:48363)
0808 23:31:15.795516 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:15.795581 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 23:31:15.796723 2899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:15.796397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.66:36863)
0808 23:31:15.796533 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:15.796612 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 23:31:15.796965 2894 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 23:31:16.116449 2901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:16.116694 2901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:16.116824 2901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:16.123693 2901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:16.123795 2901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:16.123945 2901 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:31:16.133042 2901 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:37569
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=42609
--tserver_master_addrs=127.9.44.126:41945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:16.134150 2901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:16.135335 2901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:16.147449 2920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:16.147524 2916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:16.147493 2917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:16.148432 2901 server_base.cc:1034] running on GCE node
I20260808 23:31:16.148978 2901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:16.149823 2901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:16.151014 2901 hybrid_clock.cc:648] HybridClock initialized: now 1786231876150969 us; error 31 us; skew 500 ppm
I20260808 23:31:16.151299 2901 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:31:16 dist-test-slave-6bbx krb5kdc[872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231876, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:31:16.154182 2901 init.cc:377] Logged in from keytab as kudu/127.9.44.67@KRBTEST.COM (short username kudu)
I20260808 23:31:16.155506 2901 webserver.cc:460] Webserver started at http://127.9.44.67:42609/ using document root <none> and password file <none>
I20260808 23:31:16.156138 2901 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:16.156276 2901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:16.161069 2901 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:31:16.163604 2929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:16.164783 2901 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 23:31:16.164930 2901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "a951a5353a7e407981ce7ebc27ecabf4"
format_stamp: "Formatted at 2026-08-08 23:30:30 on dist-test-slave-6bbx"
server_key: "9e77796ec4a7a1180f15d98f6f1420df"
server_key_iv: "5c336e7edf9279c61ef79a4f89bb7b88"
server_key_version: "encryptionkey@0"
I20260808 23:31:16.165828 2901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:16.181396 2901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:16.184808 2901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:16.185037 2901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:16.185899 2901 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:31:16.187139 2901 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:31:16.187242 2901 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:16.187343 2901 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:31:16.187415 2901 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:16.216500 2901 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:37569
I20260808 23:31:16.216562 3083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:37569 every 8 connection(s)
I20260808 23:31:16.217872 2901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:31:16.222047 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 2901
I20260808 23:31:16.222484 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:16.220636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.67:40853)
0808 23:31:16.221375 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:16.221579 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":263,"threads_started":1}
W20260808 23:31:16.223183 3084 heartbeater.cc:644] Failed to heartbeat to 127.9.44.126:41945 (0 consecutive failures): Network error: Failed to ping master at 127.9.44.126:41945: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
I20260808 23:31:16.223973 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:16.223638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.67:43867)
0808 23:31:16.223800 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:16.223875 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 23:31:16.224713 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:16.224434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.67:48483)
0808 23:31:16.224542 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:16.224611 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 23:31:16.224946 3084 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 23:31:16.358184 3089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:16.357465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.65:56625)
0808 23:31:16.357888 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:16.357968 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":205,"threads_started":1}
I20260808 23:31:16.798143 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:16.797412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.66:41433)
0808 23:31:16.797827 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:16.797924 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":199,"threads_started":1}
I20260808 23:31:17.226126 3091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:17.225393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.67:44589)
0808 23:31:17.225820 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:17.225918 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":201,"threads_started":1}
I20260808 23:31:17.359608 3093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:17.358932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.65:60589)
0808 23:31:17.359377 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:17.359452 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":238,"threads_started":1}
I20260808 23:31:17.799580 3094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:17.798894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.66:44621)
0808 23:31:17.799326 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:17.799416 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":213,"threads_started":1}
I20260808 23:31:18.227571 3095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:18.226907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.67:34965)
0808 23:31:18.227332 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:18.227410 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":207,"threads_started":1}
I20260808 23:31:18.361027 3097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:18.360375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.65:58671)
0808 23:31:18.360780 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:18.360855 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":205,"threads_started":1}
I20260808 23:31:18.800994 3099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:18.800315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.66:60979)
0808 23:31:18.800728 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:18.800805 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":206,"threads_started":1}
I20260808 23:31:19.228978 3102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:19.228321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.67:59423)
0808 23:31:19.228723 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:19.228799 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":208,"threads_started":1}
I20260808 23:31:19.363855 3103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:19.361855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.65:52847)
0808 23:31:19.363513 (+ 1658us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:19.363608 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1540,"thread_start_us":1419,"threads_started":1}
I20260808 23:31:19.802578 3104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:19.801814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.66:55157)
0808 23:31:19.802262 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:19.802340 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":217,"threads_started":1}
I20260808 23:31:20.230499 3105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:20.229795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.67:41479)
0808 23:31:20.230222 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:20.230302 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":191,"threads_started":1}
I20260808 23:31:20.365270 3106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:20.364612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.65:43703)
0808 23:31:20.365035 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:20.365112 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":210,"threads_started":1}
I20260808 23:31:20.804116 3107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:20.803389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.66:44183)
0808 23:31:20.803865 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:20.803951 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":222,"threads_started":1}
I20260808 23:31:21.232188 3108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:21.231402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.67:51393)
0808 23:31:21.231927 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:21.232016 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":288,"threads_started":1}
I20260808 23:31:21.367596 3109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:21.366123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.65:51271)
0808 23:31:21.367330 (+ 1207us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:21.367435 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1102,"thread_start_us":983,"threads_started":1}
I20260808 23:31:21.805747 3111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:21.805027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.66:47209)
0808 23:31:21.805477 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:21.805572 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":216,"threads_started":1}
I20260808 23:31:22.233697 3112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:22.233000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.67:58531)
0808 23:31:22.233447 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:22.233541 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":220,"threads_started":1}
I20260808 23:31:22.369164 3113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:22.368391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.65:44579)
0808 23:31:22.368886 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:22.368960 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":225,"threads_started":1}
I20260808 23:31:22.807348 3115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:22.806546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.66:39287)
0808 23:31:22.807047 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:22.807126 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":227,"threads_started":1}
I20260808 23:31:23.235265 3116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:23.234492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.67:43633)
0808 23:31:23.234969 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:23.235046 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":204,"threads_started":1}
I20260808 23:31:23.371075 3117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:23.369989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.65:37419)
0808 23:31:23.370822 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:23.370906 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":727,"thread_start_us":610,"threads_started":1}
I20260808 23:31:23.808840 3118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:23.808121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.66:48029)
0808 23:31:23.808595 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:23.808675 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":254,"threads_started":1}
I20260808 23:31:24.236770 3119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:24.236082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.67:49803)
0808 23:31:24.236508 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:24.236587 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":214,"threads_started":1}
I20260808 23:31:24.372550 3120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:24.371860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.65:57601)
0808 23:31:24.372294 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:24.372378 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":210,"threads_started":1}
I20260808 23:31:24.810271 3121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:24.809593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.66:45571)
0808 23:31:24.810027 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:24.810104 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":209,"threads_started":1}
I20260808 23:31:25.238272 3122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:25.237572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.67:49117)
0808 23:31:25.238020 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:25.238097 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":218,"threads_started":1}
I20260808 23:31:25.374143 3123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:25.373370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.65:52971)
0808 23:31:25.373910 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:25.373988 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":344,"threads_started":1}
I20260808 23:31:25.811859 3124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:25.811077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.66:35609)
0808 23:31:25.811549 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:25.811641 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":280,"threads_started":1}
I20260808 23:31:26.239730 3125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:26.239054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.67:41263)
0808 23:31:26.239460 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:26.239538 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":220,"threads_started":1}
I20260808 23:31:26.375710 3127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:26.374991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.65:56435)
0808 23:31:26.375407 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:26.375498 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":206,"threads_started":1}
I20260808 23:31:26.813373 3128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:26.812681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.66:49163)
0808 23:31:26.813107 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:26.813200 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":215,"threads_started":1}
I20260808 23:31:27.241221 3129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:27.240522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.67:34223)
0808 23:31:27.240951 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:27.241040 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":214,"threads_started":1}
I20260808 23:31:27.378096 3131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:27.376539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.65:51587)
0808 23:31:27.377819 (+ 1280us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:27.377932 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1151,"thread_start_us":1038,"threads_started":1}
I20260808 23:31:27.814816 3133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:27.814191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.66:40675)
0808 23:31:27.814589 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:27.814664 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":191,"threads_started":1}
I20260808 23:31:28.242709 3134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:28.242052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.67:41719)
0808 23:31:28.242478 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:28.242554 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":234,"threads_started":1}
I20260808 23:31:28.379571 3135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:28.378853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.65:56995)
0808 23:31:28.379306 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:28.379395 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":204,"threads_started":1}
I20260808 23:31:28.816344 3136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:28.815589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.66:36621)
0808 23:31:28.816066 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:28.816162 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":254,"threads_started":1}
I20260808 23:31:29.244362 3137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:29.243546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.67:55119)
0808 23:31:29.243990 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:29.244098 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":237,"threads_started":1}
I20260808 23:31:29.382056 3138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:29.380332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.65:46743)
0808 23:31:29.381800 (+ 1468us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:29.381880 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1359,"thread_start_us":1259,"threads_started":1}
I20260808 23:31:29.817822 3139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:29.817116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.66:46501)
0808 23:31:29.817556 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:29.817638 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":225,"threads_started":1}
I20260808 23:31:30.245808 3141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:30.245122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.67:46595)
0808 23:31:30.245550 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:30.245629 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":216,"threads_started":1}
I20260808 23:31:30.383805 3144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:30.383056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.65:37125)
0808 23:31:30.383471 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:30.383588 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":203,"threads_started":1}
I20260808 23:31:30.819630 3145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:30.818922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.66:34417)
0808 23:31:30.819349 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:30.819459 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":219,"threads_started":1}
I20260808 23:31:31.232131 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 2476
I20260808 23:31:31.247596 3148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:31.246747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:41945 (local address 127.9.44.67:43363)
0808 23:31:31.247246 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:31.247366 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.126:41945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":219,"threads_started":1}
I20260808 23:31:31.253298 3147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:31.233338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46908 (local address 127.9.44.65:40463)
0808 23:31:31.233936 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:31.233949 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:31.233987 (+ 38us) server_negotiation.cc:407] Connection header received
0808 23:31:31.235546 (+ 1559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:31.235555 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:31.236039 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:31.236322 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:31.237576 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:31.238798 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:31.240014 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:31.240659 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:31.240789 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:31.247953 (+ 7164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:31.247963 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:31.247979 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:31.248020 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:31.250543 (+ 2523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:31.251206 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:31.251212 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:31.251217 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:31.251374 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:31.251648 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:31.251650 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:31.251652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:31.252182 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:31.252406 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:31.252721 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 23:31:31.253027 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":311,"threads_started":1}
I20260808 23:31:31.254598 2703 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:31.335649 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 2476
I20260808 23:31:31.358976 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 2715
I20260808 23:31:31.375275 3150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:31.359854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38496 (local address 127.9.44.66:38901)
0808 23:31:31.360354 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:31.360369 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:31.360410 (+ 41us) server_negotiation.cc:407] Connection header received
0808 23:31:31.361843 (+ 1433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:31.361852 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:31.362288 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:31.362580 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:31.363932 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:31.365243 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:31.366494 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:31.367137 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:31.367270 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:31.369825 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:31.369832 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:31.369846 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:31.369882 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:31.372596 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:31.373209 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:31.373217 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:31.373223 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:31.373390 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:31.373666 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:31.373668 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:31.373670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:31.374146 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:31.374368 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:31.374666 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 23:31:31.374994 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":220,"threads_started":1}
I20260808 23:31:31.376662 2887 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:31.456414 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 2715
I20260808 23:31:31.481122 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 2901
I20260808 23:31:31.497364 3152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:31.482052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58848 (local address 127.9.44.67:37569)
0808 23:31:31.482619 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:31.482645 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:31.482689 (+ 44us) server_negotiation.cc:407] Connection header received
0808 23:31:31.484337 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:31.484347 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:31.484775 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:31.485060 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:31.486302 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:31.487543 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:31.488826 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:31.489462 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:31.489592 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:31.491974 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:31.491982 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:31.491995 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:31.492029 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:31.494566 (+ 2537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:31.495191 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:31.495197 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:31.495203 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:31.495353 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:31.495638 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:31.495640 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:31.495642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:31.496153 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:31.496369 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:31.496705 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 23:31:31.497066 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":240,"threads_started":1}
I20260808 23:31:31.498618 3075 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:31.578485 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 2901
2026-08-08T23:31:31Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64394 ms)
[----------] 30 tests from SecurityITest (294252 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:31:31 dist-test-slave-6bbx krb5kdc[3156](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:31:31 dist-test-slave-6bbx krb5kdc[3156](info): set up 2 sockets
Aug 08 23:31:31 dist-test-slave-6bbx krb5kdc[3156](info): commencing operation
krb5kdc: starting...
W20260808 23:31:32.785100 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.001s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:31:32 dist-test-slave-6bbx krb5kdc[3156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:31:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:31:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:31:33.043627 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:38111
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:36421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:38111
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:31:33.368650 3175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:33.368959 3175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:33.369104 3175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:33.376775 3175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:31:33.376894 3175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:33.376981 3175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:33.377060 3175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:31:33.377120 3175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:31:33.386569 3175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:36421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:38111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:38111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:33.387821 3175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:33.389052 3175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:33.400178 3184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:33.400255 3182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:33.400238 3187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:33.400987 3175 server_base.cc:1034] running on GCE node
I20260808 23:31:33.401671 3175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:33.402947 3175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:33.404140 3175 hybrid_clock.cc:648] HybridClock initialized: now 1786231893404096 us; error 30 us; skew 500 ppm
I20260808 23:31:33.404479 3175 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:31:33 dist-test-slave-6bbx krb5kdc[3156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231893, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:31:33.407371 3175 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:31:33.408797 3175 webserver.cc:460] Webserver started at http://127.9.44.126:35119/ using document root <none> and password file <none>
I20260808 23:31:33.409444 3175 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:33.409556 3175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:33.409844 3175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:33.412744 3175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "797b8309e0ff4cd19ab74a8c1e46e212"
format_stamp: "Formatted at 2026-08-08 23:31:33 on dist-test-slave-6bbx"
server_key: "cee5a13680cf8cf95d38577a447c9dc9"
server_key_iv: "8df1ca43a69c246e31c727e1995e90b6"
server_key_version: "encryptionkey@0"
I20260808 23:31:33.413475 3175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "797b8309e0ff4cd19ab74a8c1e46e212"
format_stamp: "Formatted at 2026-08-08 23:31:33 on dist-test-slave-6bbx"
server_key: "cee5a13680cf8cf95d38577a447c9dc9"
server_key_iv: "8df1ca43a69c246e31c727e1995e90b6"
server_key_version: "encryptionkey@0"
I20260808 23:31:33.418658 3175 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 23:31:33.421921 3197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:33.423151 3175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 23:31:33.423314 3175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "797b8309e0ff4cd19ab74a8c1e46e212"
format_stamp: "Formatted at 2026-08-08 23:31:33 on dist-test-slave-6bbx"
server_key: "cee5a13680cf8cf95d38577a447c9dc9"
server_key_iv: "8df1ca43a69c246e31c727e1995e90b6"
server_key_version: "encryptionkey@0"
I20260808 23:31:33.423494 3175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:33.438756 3175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:33.442269 3175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:33.442515 3175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:33.459681 3175 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:38111
I20260808 23:31:33.459774 3269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:38111 every 8 connection(s)
I20260808 23:31:33.461015 3175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:31:33.464534 3270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:33.465767 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 3175
I20260808 23:31:33.465945 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:31:33.466591 9393 external_mini_cluster.cc:1428] Setting key e4cf8b1caae5a6d377127d506e56b7e3
I20260808 23:31:33.477780 3270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 797b8309e0ff4cd19ab74a8c1e46e212: Bootstrap starting.
Aug 08 23:31:33 dist-test-slave-6bbx krb5kdc[3156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:31:33.481642 3270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 797b8309e0ff4cd19ab74a8c1e46e212: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:33.483001 3270 log.cc:824] T 00000000000000000000000000000000 P 797b8309e0ff4cd19ab74a8c1e46e212: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:33.486187 3270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 797b8309e0ff4cd19ab74a8c1e46e212: No bootstrap required, opened a new log
I20260808 23:31:33.491181 3276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:33.468519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56848 (local address 127.9.44.126:38111)
0808 23:31:33.469319 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:33.469331 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:33.469373 (+ 42us) server_negotiation.cc:407] Connection header received
0808 23:31:33.470951 (+ 1578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:33.470974 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:33.471329 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:33.471619 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:33.473440 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:33.474753 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:33.476688 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:33.477321 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:33.477447 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:33.482955 (+ 5508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:33.482971 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:33.482995 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:33.483085 (+ 90us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:33.487156 (+ 4071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:33.488027 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:33.488034 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:33.488043 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:33.488219 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:33.488601 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:33.488604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:33.488606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:33.489190 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:33.489418 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:33.489826 (+ 408us) server_negotiation.cc:299] Negotiation successful
0808 23:31:33.490167 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":384,"threads_started":1}
I20260808 23:31:33.494611 3270 raft_consensus.cc:348] T 00000000000000000000000000000000 P 797b8309e0ff4cd19ab74a8c1e46e212 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "797b8309e0ff4cd19ab74a8c1e46e212" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 38111 } }
I20260808 23:31:33.494935 3270 raft_consensus.cc:374] T 00000000000000000000000000000000 P 797b8309e0ff4cd19ab74a8c1e46e212 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:33.495026 3270 raft_consensus.cc:729] T 00000000000000000000000000000000 P 797b8309e0ff4cd19ab74a8c1e46e212 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 797b8309e0ff4cd19ab74a8c1e46e212, State: Initialized, Role: FOLLOWER
I20260808 23:31:33.495530 3270 consensus_queue.cc:260] T 00000000000000000000000000000000 P 797b8309e0ff4cd19ab74a8c1e46e212 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "797b8309e0ff4cd19ab74a8c1e46e212" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 38111 } }
I20260808 23:31:33.495774 3270 raft_consensus.cc:388] T 00000000000000000000000000000000 P 797b8309e0ff4cd19ab74a8c1e46e212 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:31:33.495898 3270 raft_consensus.cc:482] T 00000000000000000000000000000000 P 797b8309e0ff4cd19ab74a8c1e46e212 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:31:33.496045 3270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 797b8309e0ff4cd19ab74a8c1e46e212 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:33.498072 3270 raft_consensus.cc:504] T 00000000000000000000000000000000 P 797b8309e0ff4cd19ab74a8c1e46e212 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "797b8309e0ff4cd19ab74a8c1e46e212" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 38111 } }
I20260808 23:31:33.498548 3270 leader_election.cc:302] T 00000000000000000000000000000000 P 797b8309e0ff4cd19ab74a8c1e46e212 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 797b8309e0ff4cd19ab74a8c1e46e212; no voters:
I20260808 23:31:33.499065 3270 leader_election.cc:288] T 00000000000000000000000000000000 P 797b8309e0ff4cd19ab74a8c1e46e212 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:31:33.499187 3279 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 797b8309e0ff4cd19ab74a8c1e46e212 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:31:33.499821 3279 raft_consensus.cc:686] T 00000000000000000000000000000000 P 797b8309e0ff4cd19ab74a8c1e46e212 [term 1 LEADER]: Becoming Leader. State: Replica: 797b8309e0ff4cd19ab74a8c1e46e212, State: Running, Role: LEADER
I20260808 23:31:33.500351 3279 consensus_queue.cc:237] T 00000000000000000000000000000000 P 797b8309e0ff4cd19ab74a8c1e46e212 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "797b8309e0ff4cd19ab74a8c1e46e212" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 38111 } }
I20260808 23:31:33.501009 3270 sys_catalog.cc:564] T 00000000000000000000000000000000 P 797b8309e0ff4cd19ab74a8c1e46e212 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:31:33.503015 3280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 797b8309e0ff4cd19ab74a8c1e46e212 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "797b8309e0ff4cd19ab74a8c1e46e212" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "797b8309e0ff4cd19ab74a8c1e46e212" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 38111 } } }
I20260808 23:31:33.503319 3280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 797b8309e0ff4cd19ab74a8c1e46e212 [sys.catalog]: This master's current role is: LEADER
I20260808 23:31:33.504139 3291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:31:33.504032 3281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 797b8309e0ff4cd19ab74a8c1e46e212 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 797b8309e0ff4cd19ab74a8c1e46e212. Latest consensus state: current_term: 1 leader_uuid: "797b8309e0ff4cd19ab74a8c1e46e212" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "797b8309e0ff4cd19ab74a8c1e46e212" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 38111 } } }
I20260808 23:31:33.504340 3281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 797b8309e0ff4cd19ab74a8c1e46e212 [sys.catalog]: This master's current role is: LEADER
I20260808 23:31:33.508876 3291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:31:33.521080 3291 catalog_manager.cc:1347] Generated new cluster ID: d868551bb2904d96b90821fa430233c7
I20260808 23:31:33.521191 3291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:31:33.538883 3291 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:31:33.540052 3291 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:31:33.547524 3291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 797b8309e0ff4cd19ab74a8c1e46e212: Generated new TSK 0
I20260808 23:31:33.548295 3291 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.44.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
I20260808 23:31:33.621286 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:0
--local_ip_for_outbound_sockets=127.9.44.65
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:38111
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:36421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:31:33.970239 3310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:33.970494 3310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:33.970623 3310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:33.977464 3310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:33.977571 3310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:33.977718 3310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:31:33.986881 3310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:36421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:38111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:33.988009 3310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:33.989202 3310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:34.001402 3315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:34.001375 3317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:34.001437 3322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:34.002380 3310 server_base.cc:1034] running on GCE node
I20260808 23:31:34.002991 3310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:34.003899 3310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:34.005102 3310 hybrid_clock.cc:648] HybridClock initialized: now 1786231894005055 us; error 40 us; skew 500 ppm
I20260808 23:31:34.005412 3310 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:31:34 dist-test-slave-6bbx krb5kdc[3156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231894, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:31:34.008440 3310 init.cc:377] Logged in from keytab as kudu/127.9.44.65@KRBTEST.COM (short username kudu)
I20260808 23:31:34.009765 3310 webserver.cc:460] Webserver started at http://127.9.44.65:37969/ using document root <none> and password file <none>
I20260808 23:31:34.010418 3310 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:34.010538 3310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:34.010828 3310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:34.013679 3310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-0/data/instance:
uuid: "c5843bb7158442289c055d82b1b4110a"
format_stamp: "Formatted at 2026-08-08 23:31:34 on dist-test-slave-6bbx"
server_key: "d853055ecaa37898137de5d1212951e1"
server_key_iv: "2d6e4ae61db3d27e80d7b8a0571f90e8"
server_key_version: "encryptionkey@0"
I20260808 23:31:34.014444 3310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance:
uuid: "c5843bb7158442289c055d82b1b4110a"
format_stamp: "Formatted at 2026-08-08 23:31:34 on dist-test-slave-6bbx"
server_key: "d853055ecaa37898137de5d1212951e1"
server_key_iv: "2d6e4ae61db3d27e80d7b8a0571f90e8"
server_key_version: "encryptionkey@0"
I20260808 23:31:34.019344 3310 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260808 23:31:34.022608 3329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:34.023818 3310 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 23:31:34.023970 3310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "c5843bb7158442289c055d82b1b4110a"
format_stamp: "Formatted at 2026-08-08 23:31:34 on dist-test-slave-6bbx"
server_key: "d853055ecaa37898137de5d1212951e1"
server_key_iv: "2d6e4ae61db3d27e80d7b8a0571f90e8"
server_key_version: "encryptionkey@0"
I20260808 23:31:34.024152 3310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:34.052309 3310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:34.055814 3310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:34.056079 3310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:34.056931 3310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:31:34.058163 3310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:31:34.058246 3310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:34.058363 3310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:31:34.058426 3310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:34.087245 3310 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:43299
I20260808 23:31:34.087302 3499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:43299 every 8 connection(s)
I20260808 23:31:34.088467 3310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:31:34.092993 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 3310
I20260808 23:31:34.093161 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance
I20260808 23:31:34.093786 9393 external_mini_cluster.cc:1428] Setting key f2792f74e08952b23957cffb0b037bcb
Aug 08 23:31:34 dist-test-slave-6bbx krb5kdc[3156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231894, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:31:34.108044 3502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:34.090769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:55925 (local address 127.9.44.126:38111)
0808 23:31:34.091215 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:34.091221 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:34.092111 (+ 890us) server_negotiation.cc:407] Connection header received
0808 23:31:34.093940 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:34.093947 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:34.094032 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:34.094153 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:34.095735 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:34.096458 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:34.098549 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:34.098945 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:34.099039 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:34.102747 (+ 3708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:34.102752 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:34.102755 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:34.102788 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:34.105204 (+ 2416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:34.105856 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:34.105861 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:34.105863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:34.105985 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:34.106392 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:34.106397 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:34.106400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:34.106661 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:34.106801 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:34.107447 (+ 646us) server_negotiation.cc:299] Negotiation successful
0808 23:31:34.107697 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":208,"threads_started":1}
I20260808 23:31:34.108836 3503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:34.091284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:38111 (local address 127.9.44.65:55925)
0808 23:31:34.091962 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:34.092001 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:34.093661 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:34.094323 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:34.094330 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:34.094932 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:31:34.095541 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:34.095568 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:34.096618 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:34.096623 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:34.098422 (+ 1799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:34.098440 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:34.099155 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:34.099161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:34.099444 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:34.100092 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:31:34.100117 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:31:34.102534 (+ 2417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:31:34.105335 (+ 2801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:31:34.105343 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:31:34.105358 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:31:34.105744 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:31:34.106088 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:31:34.106093 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:31:34.106097 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:31:34.106287 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:31:34.106791 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:31:34.106798 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:31:34.107086 (+ 288us) client_negotiation.cc:769] Sending connection context
0808 23:31:34.107318 (+ 232us) client_negotiation.cc:240] Negotiation successful
0808 23:31:34.107814 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":253,"threads_started":1}
I20260808 23:31:34.110283 3500 heartbeater.cc:347] Connected to a master server at 127.9.44.126:38111
I20260808 23:31:34.110577 3500 heartbeater.cc:464] Registering TS with master...
I20260808 23:31:34.111354 3500 heartbeater.cc:511] Master 127.9.44.126:38111 requested a full tablet report, sending...
I20260808 23:31:34.113468 3220 ts_manager.cc:194] Registered new tserver with Master: c5843bb7158442289c055d82b1b4110a (127.9.44.65:43299)
I20260808 23:31:34.114862 3220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.65@KRBTEST.COM'} at 127.9.44.65:55925
WARNING: no policy specified for kudu/127.9.44.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
I20260808 23:31:34.155848 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:0
--local_ip_for_outbound_sockets=127.9.44.66
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:38111
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:36421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:31:34.476320 3507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:34.476583 3507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:34.476713 3507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:34.483800 3507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:34.483911 3507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:34.484057 3507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:31:34.493287 3507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:36421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:38111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:34.494475 3507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:34.495757 3507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:34.507858 3514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:34.507908 3515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:34.508074 3517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:34.508921 3507 server_base.cc:1034] running on GCE node
I20260808 23:31:34.509471 3507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:34.510308 3507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:34.511512 3507 hybrid_clock.cc:648] HybridClock initialized: now 1786231894511463 us; error 40 us; skew 500 ppm
I20260808 23:31:34.511831 3507 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:31:34 dist-test-slave-6bbx krb5kdc[3156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231894, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:31:34.514853 3507 init.cc:377] Logged in from keytab as kudu/127.9.44.66@KRBTEST.COM (short username kudu)
I20260808 23:31:34.516350 3507 webserver.cc:460] Webserver started at http://127.9.44.66:39467/ using document root <none> and password file <none>
I20260808 23:31:34.517006 3507 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:34.517123 3507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:34.517421 3507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:34.520222 3507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-1/data/instance:
uuid: "b6e7d4bed9b44098a8a2a4692b745926"
format_stamp: "Formatted at 2026-08-08 23:31:34 on dist-test-slave-6bbx"
server_key: "66a252d79371aaad5b993a947d627d87"
server_key_iv: "56dbc4c0e85d656c8f9ec7e20080dc2c"
server_key_version: "encryptionkey@0"
I20260808 23:31:34.520993 3507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance:
uuid: "b6e7d4bed9b44098a8a2a4692b745926"
format_stamp: "Formatted at 2026-08-08 23:31:34 on dist-test-slave-6bbx"
server_key: "66a252d79371aaad5b993a947d627d87"
server_key_iv: "56dbc4c0e85d656c8f9ec7e20080dc2c"
server_key_version: "encryptionkey@0"
I20260808 23:31:34.525985 3507 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 23:31:34.529481 3527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:34.530754 3507 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 23:31:34.530910 3507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "b6e7d4bed9b44098a8a2a4692b745926"
format_stamp: "Formatted at 2026-08-08 23:31:34 on dist-test-slave-6bbx"
server_key: "66a252d79371aaad5b993a947d627d87"
server_key_iv: "56dbc4c0e85d656c8f9ec7e20080dc2c"
server_key_version: "encryptionkey@0"
I20260808 23:31:34.531093 3507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:34.550872 3507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:34.554780 3507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:34.555040 3507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:34.555953 3507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:31:34.557178 3507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:31:34.557261 3507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:34.557379 3507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:31:34.557431 3507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:34.586357 3507 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:35559
I20260808 23:31:34.586410 3679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:35559 every 8 connection(s)
I20260808 23:31:34.587565 3507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:31:34.588879 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 3507
I20260808 23:31:34.589077 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance
I20260808 23:31:34.589620 9393 external_mini_cluster.cc:1428] Setting key 4c8878fdb95b808771b310be574857ad
Aug 08 23:31:34 dist-test-slave-6bbx krb5kdc[3156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231894, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:31:34.606395 3502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:34.590294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:39121 (local address 127.9.44.126:38111)
0808 23:31:34.590510 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:34.590516 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:34.591697 (+ 1181us) server_negotiation.cc:407] Connection header received
0808 23:31:34.593537 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:34.593543 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:34.593627 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:34.593713 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:34.595147 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:34.595887 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:34.597396 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:34.597812 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:34.597920 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:34.601165 (+ 3245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:34.601170 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:34.601174 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:34.601204 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:34.603621 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:34.604288 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:34.604293 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:34.604297 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:34.604424 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:34.604824 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:34.604829 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:34.604831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:34.605085 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:34.605226 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:34.605958 (+ 732us) server_negotiation.cc:299] Negotiation successful
0808 23:31:34.606139 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 23:31:34.607502 3682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:34.590730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:38111 (local address 127.9.44.66:39121)
0808 23:31:34.591514 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:34.591568 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:34.593265 (+ 1697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:34.593960 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:34.593968 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:34.594448 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:31:34.594984 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:34.595001 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:34.596040 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:34.596044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:34.597258 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:34.597270 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:34.598030 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:34.598038 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:34.598254 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:34.598873 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:31:34.598899 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:31:34.600963 (+ 2064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:31:34.603764 (+ 2801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:31:34.603773 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:31:34.603788 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:31:34.604174 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:31:34.604526 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:31:34.604531 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:31:34.604535 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:31:34.604718 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:31:34.605207 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:31:34.605215 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:31:34.605528 (+ 313us) client_negotiation.cc:769] Sending connection context
0808 23:31:34.605825 (+ 297us) client_negotiation.cc:240] Negotiation successful
0808 23:31:34.606291 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":284,"threads_started":1}
I20260808 23:31:34.608839 3680 heartbeater.cc:347] Connected to a master server at 127.9.44.126:38111
I20260808 23:31:34.609163 3680 heartbeater.cc:464] Registering TS with master...
I20260808 23:31:34.609961 3680 heartbeater.cc:511] Master 127.9.44.126:38111 requested a full tablet report, sending...
I20260808 23:31:34.611449 3220 ts_manager.cc:194] Registered new tserver with Master: b6e7d4bed9b44098a8a2a4692b745926 (127.9.44.66:35559)
I20260808 23:31:34.612278 3220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.66@KRBTEST.COM'} at 127.9.44.66:39121
WARNING: no policy specified for kudu/127.9.44.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
I20260808 23:31:34.653052 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:0
--local_ip_for_outbound_sockets=127.9.44.67
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:38111
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:36421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:31:34.986474 3685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:34.986748 3685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:34.986871 3685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:34.993852 3685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:34.993968 3685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:34.994117 3685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:31:35.003532 3685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:36421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:38111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:35.004696 3685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:35.005920 3685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:35.018205 3693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:35.018122 3695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:35.018169 3697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:35.019434 3685 server_base.cc:1034] running on GCE node
I20260808 23:31:35.020004 3685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:35.020804 3685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:35.021991 3685 hybrid_clock.cc:648] HybridClock initialized: now 1786231895021942 us; error 41 us; skew 500 ppm
I20260808 23:31:35.022292 3685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:31:35 dist-test-slave-6bbx krb5kdc[3156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231895, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:31:35.025359 3685 init.cc:377] Logged in from keytab as kudu/127.9.44.67@KRBTEST.COM (short username kudu)
I20260808 23:31:35.026652 3685 webserver.cc:460] Webserver started at http://127.9.44.67:42007/ using document root <none> and password file <none>
I20260808 23:31:35.027400 3685 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:35.027522 3685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:35.027904 3685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:35.030800 3685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-2/data/instance:
uuid: "2fa66641a264448f8f8e82f506f7ebed"
format_stamp: "Formatted at 2026-08-08 23:31:35 on dist-test-slave-6bbx"
server_key: "5a7e75f3839b460ca2c130d6f047b9ed"
server_key_iv: "b0d3a9a3b5cc83a53507d794ce3e1a03"
server_key_version: "encryptionkey@0"
I20260808 23:31:35.031507 3685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance:
uuid: "2fa66641a264448f8f8e82f506f7ebed"
format_stamp: "Formatted at 2026-08-08 23:31:35 on dist-test-slave-6bbx"
server_key: "5a7e75f3839b460ca2c130d6f047b9ed"
server_key_iv: "b0d3a9a3b5cc83a53507d794ce3e1a03"
server_key_version: "encryptionkey@0"
I20260808 23:31:35.036445 3685 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.003s
I20260808 23:31:35.039918 3705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:35.041138 3685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 23:31:35.041304 3685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "2fa66641a264448f8f8e82f506f7ebed"
format_stamp: "Formatted at 2026-08-08 23:31:35 on dist-test-slave-6bbx"
server_key: "5a7e75f3839b460ca2c130d6f047b9ed"
server_key_iv: "b0d3a9a3b5cc83a53507d794ce3e1a03"
server_key_version: "encryptionkey@0"
I20260808 23:31:35.041481 3685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:35.086294 3685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:35.090690 3685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:35.090961 3685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:35.091945 3685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:31:35.093251 3685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:31:35.093333 3685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:35.093451 3685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:31:35.093505 3685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:35.118111 3500 heartbeater.cc:503] Master 127.9.44.126:38111 was elected leader, sending a full tablet report...
I20260808 23:31:35.124081 3685 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:39999
I20260808 23:31:35.124152 3871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:39999 every 8 connection(s)
I20260808 23:31:35.125308 3685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:31:35.127765 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 3685
I20260808 23:31:35.127954 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance
I20260808 23:31:35.128520 9393 external_mini_cluster.cc:1428] Setting key 70545fd9a9b16c2688eb1afcda6d93c7
Aug 08 23:31:35 dist-test-slave-6bbx krb5kdc[3156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231895, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:31:35.144997 3874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:35.127936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:43255 (local address 127.9.44.126:38111)
0808 23:31:35.128458 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:35.128465 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:35.129377 (+ 912us) server_negotiation.cc:407] Connection header received
0808 23:31:35.131618 (+ 2241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:35.131622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:35.131716 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:35.131805 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:35.133240 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.134016 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.135571 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.136010 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.136106 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:35.139305 (+ 3199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:35.139310 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:35.139313 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:35.139344 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:35.141765 (+ 2421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:35.142518 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:35.142522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:35.142524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:35.142647 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:35.143283 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:35.143287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:35.143289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:35.143555 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:35.143699 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:35.144389 (+ 690us) server_negotiation.cc:299] Negotiation successful
0808 23:31:35.144604 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":267,"threads_started":1}
I20260808 23:31:35.145852 3875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:35.128396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:38111 (local address 127.9.44.67:43255)
0808 23:31:35.129225 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:35.129264 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:35.131356 (+ 2092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:35.131996 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:35.132004 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:35.132495 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:31:35.133057 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:35.133075 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.134138 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.134143 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:35.135438 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:35.135451 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.136181 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.136186 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:35.136376 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:35.136979 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:31:35.137001 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:31:35.139112 (+ 2111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:31:35.141893 (+ 2781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:31:35.141901 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:31:35.141916 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:31:35.142405 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:31:35.142760 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:31:35.142765 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:31:35.142769 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:31:35.142950 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:31:35.143715 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:31:35.143723 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:31:35.144012 (+ 289us) client_negotiation.cc:769] Sending connection context
0808 23:31:35.144240 (+ 228us) client_negotiation.cc:240] Negotiation successful
0808 23:31:35.144770 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":313,"threads_started":1}
I20260808 23:31:35.147069 3872 heartbeater.cc:347] Connected to a master server at 127.9.44.126:38111
I20260808 23:31:35.147346 3872 heartbeater.cc:464] Registering TS with master...
I20260808 23:31:35.148070 3872 heartbeater.cc:511] Master 127.9.44.126:38111 requested a full tablet report, sending...
I20260808 23:31:35.149446 3220 ts_manager.cc:194] Registered new tserver with Master: 2fa66641a264448f8f8e82f506f7ebed (127.9.44.67:39999)
I20260808 23:31:35.150204 3220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.67@KRBTEST.COM'} at 127.9.44.67:43255
I20260808 23:31:35.155426 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:31:35.171298 3874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:35.159952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56856 (local address 127.9.44.126:38111)
0808 23:31:35.160141 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:35.160147 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:35.160581 (+ 434us) server_negotiation.cc:407] Connection header received
0808 23:31:35.160699 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:35.160702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:35.160759 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:35.160832 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:35.162241 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.162927 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.164194 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.164576 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.164675 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:35.166837 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:35.166842 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:35.166845 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:35.166874 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:35.169113 (+ 2239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:35.169714 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:35.169718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:35.169720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:35.169827 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:35.170104 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:35.170107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:35.170109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:35.170385 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:35.170640 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:35.170861 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 23:31:35.171018 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 23:31:35.174800 3220 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56856:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:31:35.181504 3220 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:31:35.219942 3889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:35.192807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:43299 (local address 127.0.0.1:47174)
0808 23:31:35.204892 (+ 12085us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:35.204931 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:35.205129 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:35.209446 (+ 4317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:35.209454 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:35.209483 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:35.209871 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:35.209884 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.212965 (+ 3081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.212971 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:35.216660 (+ 3689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:35.216715 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.218912 (+ 2197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.218918 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:35.219114 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:35.219165 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 23:31:35.219313 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 23:31:35.219487 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11847,"mutex_wait_us":1,"thread_start_us":12486,"threads_started":1}
I20260808 23:31:35.220927 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:35.193024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47174 (local address 127.9.44.65:43299)
0808 23:31:35.208816 (+ 15792us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:35.208825 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:35.208856 (+ 31us) server_negotiation.cc:407] Connection header received
0808 23:31:35.208973 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:35.208981 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:35.209221 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:35.209437 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:35.210068 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.212799 (+ 2731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.216918 (+ 4119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.218410 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.218537 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:35.218776 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:35.220427 (+ 1651us) server_negotiation.cc:299] Negotiation successful
0808 23:31:35.220607 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15614,"thread_start_us":16576,"threads_started":1}
I20260808 23:31:35.221259 3896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:35.193883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40946 (local address 127.9.44.67:39999)
0808 23:31:35.210271 (+ 16388us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:35.210280 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:35.212109 (+ 1829us) server_negotiation.cc:407] Connection header received
0808 23:31:35.212326 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:35.212334 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:35.212547 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:35.212735 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:35.214015 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.215594 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.218261 (+ 2667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.219717 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.219834 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:35.220056 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:35.220722 (+ 666us) server_negotiation.cc:299] Negotiation successful
0808 23:31:35.220905 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16221,"thread_start_us":16062,"threads_started":1}
I20260808 23:31:35.221312 3891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:35.193039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:35559 (local address 127.0.0.1:41936)
0808 23:31:35.208107 (+ 15068us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:35.208126 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:35.208243 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:35.212255 (+ 4012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:35.212259 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:35.212283 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:35.214265 (+ 1982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:35.214280 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.216596 (+ 2316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.216608 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:35.218734 (+ 2126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:35.218749 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.220551 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.220557 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:35.220764 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:35.220794 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 23:31:35.220857 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 23:31:35.220959 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14953,"mutex_wait_us":20,"thread_start_us":14814,"threads_started":1}
I20260808 23:31:35.222271 3893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:35.193255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:39999 (local address 127.0.0.1:40946)
0808 23:31:35.211996 (+ 18741us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:35.212010 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:35.212130 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:35.213076 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:35.213082 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:35.213111 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:35.213602 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:35.213617 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.215841 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.215846 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:35.218080 (+ 2234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:35.218096 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.220194 (+ 2098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.220201 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:35.220369 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:35.220402 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 23:31:35.220571 (+ 169us) client_negotiation.cc:240] Negotiation successful
0808 23:31:35.221881 (+ 1310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18641,"mutex_wait_us":1166,"thread_start_us":18505,"threads_started":1}
I20260808 23:31:35.222344 3894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:35.193624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41936 (local address 127.9.44.66:35559)
0808 23:31:35.210767 (+ 17143us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:35.210775 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:35.210806 (+ 31us) server_negotiation.cc:407] Connection header received
0808 23:31:35.210970 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:35.210979 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:35.211214 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:35.211400 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:35.214561 (+ 3161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.216431 (+ 1870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.218927 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.220329 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.221590 (+ 1261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:35.221774 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:35.221845 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 23:31:35.222011 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16916,"spinlock_wait_cycles":24448,"thread_start_us":16870,"threads_started":1}
I20260808 23:31:35.225419 3594 tablet_service.cc:1467] Processing CreateTablet for tablet 49b84ed65d0a43b9ad85a3b53a1e15f2 (DEFAULT_TABLE table=test-table [id=c871001570714995b82f3359c1aaf13c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:31:35.225615 3410 tablet_service.cc:1467] Processing CreateTablet for tablet 49b84ed65d0a43b9ad85a3b53a1e15f2 (DEFAULT_TABLE table=test-table [id=c871001570714995b82f3359c1aaf13c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:31:35.225919 3775 tablet_service.cc:1467] Processing CreateTablet for tablet 49b84ed65d0a43b9ad85a3b53a1e15f2 (DEFAULT_TABLE table=test-table [id=c871001570714995b82f3359c1aaf13c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:31:35.226691 3594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49b84ed65d0a43b9ad85a3b53a1e15f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:35.227048 3410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49b84ed65d0a43b9ad85a3b53a1e15f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:35.227247 3775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49b84ed65d0a43b9ad85a3b53a1e15f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:35.237885 3897 tablet_bootstrap.cc:492] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926: Bootstrap starting.
I20260808 23:31:35.241065 3898 tablet_bootstrap.cc:492] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P c5843bb7158442289c055d82b1b4110a: Bootstrap starting.
I20260808 23:31:35.241250 3897 tablet_bootstrap.cc:654] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:35.242553 3899 tablet_bootstrap.cc:492] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P 2fa66641a264448f8f8e82f506f7ebed: Bootstrap starting.
I20260808 23:31:35.244321 3897 log.cc:824] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:35.244598 3898 tablet_bootstrap.cc:654] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P c5843bb7158442289c055d82b1b4110a: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:35.245587 3899 tablet_bootstrap.cc:654] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P 2fa66641a264448f8f8e82f506f7ebed: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:35.245723 3898 log.cc:824] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P c5843bb7158442289c055d82b1b4110a: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:35.246722 3899 log.cc:824] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P 2fa66641a264448f8f8e82f506f7ebed: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:35.247542 3897 tablet_bootstrap.cc:492] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926: No bootstrap required, opened a new log
I20260808 23:31:35.247855 3897 ts_tablet_manager.cc:1397] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260808 23:31:35.248741 3898 tablet_bootstrap.cc:492] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P c5843bb7158442289c055d82b1b4110a: No bootstrap required, opened a new log
I20260808 23:31:35.249015 3898 ts_tablet_manager.cc:1397] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P c5843bb7158442289c055d82b1b4110a: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.004s
I20260808 23:31:35.249100 3899 tablet_bootstrap.cc:492] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P 2fa66641a264448f8f8e82f506f7ebed: No bootstrap required, opened a new log
I20260808 23:31:35.249323 3899 ts_tablet_manager.cc:1397] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P 2fa66641a264448f8f8e82f506f7ebed: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260808 23:31:35.256263 3899 raft_consensus.cc:348] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P 2fa66641a264448f8f8e82f506f7ebed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5843bb7158442289c055d82b1b4110a" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43299 } } peers { permanent_uuid: "b6e7d4bed9b44098a8a2a4692b745926" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 35559 } } peers { permanent_uuid: "2fa66641a264448f8f8e82f506f7ebed" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39999 } }
I20260808 23:31:35.256590 3899 raft_consensus.cc:374] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P 2fa66641a264448f8f8e82f506f7ebed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:35.256691 3899 raft_consensus.cc:729] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P 2fa66641a264448f8f8e82f506f7ebed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fa66641a264448f8f8e82f506f7ebed, State: Initialized, Role: FOLLOWER
I20260808 23:31:35.257272 3899 consensus_queue.cc:260] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P 2fa66641a264448f8f8e82f506f7ebed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5843bb7158442289c055d82b1b4110a" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43299 } } peers { permanent_uuid: "b6e7d4bed9b44098a8a2a4692b745926" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 35559 } } peers { permanent_uuid: "2fa66641a264448f8f8e82f506f7ebed" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39999 } }
I20260808 23:31:35.258353 3872 heartbeater.cc:503] Master 127.9.44.126:38111 was elected leader, sending a full tablet report...
I20260808 23:31:35.258797 3899 ts_tablet_manager.cc:1428] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P 2fa66641a264448f8f8e82f506f7ebed: Time spent starting tablet: real 0.009s user 0.005s sys 0.005s
I20260808 23:31:35.258533 3897 raft_consensus.cc:348] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5843bb7158442289c055d82b1b4110a" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43299 } } peers { permanent_uuid: "b6e7d4bed9b44098a8a2a4692b745926" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 35559 } } peers { permanent_uuid: "2fa66641a264448f8f8e82f506f7ebed" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39999 } }
I20260808 23:31:35.258931 3897 raft_consensus.cc:374] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:35.259037 3897 raft_consensus.cc:729] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6e7d4bed9b44098a8a2a4692b745926, State: Initialized, Role: FOLLOWER
I20260808 23:31:35.259613 3897 consensus_queue.cc:260] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5843bb7158442289c055d82b1b4110a" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43299 } } peers { permanent_uuid: "b6e7d4bed9b44098a8a2a4692b745926" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 35559 } } peers { permanent_uuid: "2fa66641a264448f8f8e82f506f7ebed" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39999 } }
I20260808 23:31:35.260703 3680 heartbeater.cc:503] Master 127.9.44.126:38111 was elected leader, sending a full tablet report...
I20260808 23:31:35.260839 3898 raft_consensus.cc:348] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P c5843bb7158442289c055d82b1b4110a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5843bb7158442289c055d82b1b4110a" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43299 } } peers { permanent_uuid: "b6e7d4bed9b44098a8a2a4692b745926" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 35559 } } peers { permanent_uuid: "2fa66641a264448f8f8e82f506f7ebed" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39999 } }
I20260808 23:31:35.261260 3898 raft_consensus.cc:374] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P c5843bb7158442289c055d82b1b4110a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:35.261354 3898 raft_consensus.cc:729] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P c5843bb7158442289c055d82b1b4110a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5843bb7158442289c055d82b1b4110a, State: Initialized, Role: FOLLOWER
I20260808 23:31:35.261380 3897 ts_tablet_manager.cc:1428] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260808 23:31:35.261996 3898 consensus_queue.cc:260] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P c5843bb7158442289c055d82b1b4110a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5843bb7158442289c055d82b1b4110a" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43299 } } peers { permanent_uuid: "b6e7d4bed9b44098a8a2a4692b745926" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 35559 } } peers { permanent_uuid: "2fa66641a264448f8f8e82f506f7ebed" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39999 } }
I20260808 23:31:35.263695 3898 ts_tablet_manager.cc:1428] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P c5843bb7158442289c055d82b1b4110a: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
W20260808 23:31:35.339766 3681 tablet.cc:2369] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:31:35.340591 3501 tablet.cc:2369] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P c5843bb7158442289c055d82b1b4110a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:31:35.369068 3910 raft_consensus.cc:482] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:31:35.369306 3910 raft_consensus.cc:504] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5843bb7158442289c055d82b1b4110a" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43299 } } peers { permanent_uuid: "b6e7d4bed9b44098a8a2a4692b745926" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 35559 } } peers { permanent_uuid: "2fa66641a264448f8f8e82f506f7ebed" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39999 } }
I20260808 23:31:35.370947 3910 leader_election.cc:288] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5843bb7158442289c055d82b1b4110a (127.9.44.65:43299), 2fa66641a264448f8f8e82f506f7ebed (127.9.44.67:39999)
W20260808 23:31:35.376853 3873 tablet.cc:2369] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P 2fa66641a264448f8f8e82f506f7ebed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:31:35.377451 3896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:35.371587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:38249 (local address 127.9.44.67:39999)
0808 23:31:35.371814 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:35.371820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:35.372243 (+ 423us) server_negotiation.cc:407] Connection header received
0808 23:31:35.372445 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:35.372448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:35.372509 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:35.372581 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:35.373128 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.373864 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.375696 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.376461 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.376573 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:35.376674 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:35.377048 (+ 374us) server_negotiation.cc:299] Negotiation successful
0808 23:31:35.377182 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260808 23:31:35.377574 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:35.371392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:49511 (local address 127.9.44.65:43299)
0808 23:31:35.371591 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:35.371597 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:35.372448 (+ 851us) server_negotiation.cc:407] Connection header received
0808 23:31:35.372608 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:35.372612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:35.372670 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:35.372764 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:35.373343 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.374048 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.375840 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.376654 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.376797 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:35.376908 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:35.377210 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 23:31:35.377342 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:31:35.377627 3914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:35.371334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:43299 (local address 127.9.44.66:49511)
0808 23:31:35.372334 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:35.372349 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:35.372483 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:35.372796 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:35.372805 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:35.372830 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:35.373197 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:35.373207 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.374189 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.374193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:35.375692 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:35.375708 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.376868 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.376874 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:35.377047 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:35.377077 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 23:31:35.377155 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 23:31:35.377267 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":889,"mutex_wait_us":34,"thread_start_us":444,"threads_started":1}
I20260808 23:31:35.378113 3915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:35.371316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:39999 (local address 127.9.44.66:38249)
0808 23:31:35.372127 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:35.372146 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:35.372306 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:35.372610 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:35.372614 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:35.372639 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:35.372992 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:35.373002 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.373987 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.373991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:35.375524 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:35.375536 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.376668 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.376674 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:35.376836 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:35.376874 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 23:31:35.377678 (+ 804us) client_negotiation.cc:240] Negotiation successful
0808 23:31:35.377804 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"mutex_wait_us":1,"thread_start_us":703,"threads_started":1}
I20260808 23:31:35.378721 3436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49b84ed65d0a43b9ad85a3b53a1e15f2" candidate_uuid: "b6e7d4bed9b44098a8a2a4692b745926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5843bb7158442289c055d82b1b4110a" is_pre_election: true
I20260808 23:31:35.379124 3806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49b84ed65d0a43b9ad85a3b53a1e15f2" candidate_uuid: "b6e7d4bed9b44098a8a2a4692b745926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fa66641a264448f8f8e82f506f7ebed" is_pre_election: true
I20260808 23:31:35.379247 3436 raft_consensus.cc:2454] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P c5843bb7158442289c055d82b1b4110a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6e7d4bed9b44098a8a2a4692b745926 in term 0.
I20260808 23:31:35.379597 3806 raft_consensus.cc:2454] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P 2fa66641a264448f8f8e82f506f7ebed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6e7d4bed9b44098a8a2a4692b745926 in term 0.
I20260808 23:31:35.379997 3535 leader_election.cc:302] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6e7d4bed9b44098a8a2a4692b745926, c5843bb7158442289c055d82b1b4110a; no voters:
I20260808 23:31:35.380393 3910 raft_consensus.cc:2781] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:31:35.380551 3910 raft_consensus.cc:482] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:31:35.380656 3910 raft_consensus.cc:3037] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:35.382786 3910 raft_consensus.cc:504] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5843bb7158442289c055d82b1b4110a" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43299 } } peers { permanent_uuid: "b6e7d4bed9b44098a8a2a4692b745926" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 35559 } } peers { permanent_uuid: "2fa66641a264448f8f8e82f506f7ebed" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39999 } }
I20260808 23:31:35.383376 3910 leader_election.cc:288] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926 [CANDIDATE]: Term 1 election: Requested vote from peers c5843bb7158442289c055d82b1b4110a (127.9.44.65:43299), 2fa66641a264448f8f8e82f506f7ebed (127.9.44.67:39999)
I20260808 23:31:35.383741 3436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49b84ed65d0a43b9ad85a3b53a1e15f2" candidate_uuid: "b6e7d4bed9b44098a8a2a4692b745926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5843bb7158442289c055d82b1b4110a"
I20260808 23:31:35.383844 3806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49b84ed65d0a43b9ad85a3b53a1e15f2" candidate_uuid: "b6e7d4bed9b44098a8a2a4692b745926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fa66641a264448f8f8e82f506f7ebed"
I20260808 23:31:35.383926 3436 raft_consensus.cc:3037] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P c5843bb7158442289c055d82b1b4110a [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:35.384035 3806 raft_consensus.cc:3037] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P 2fa66641a264448f8f8e82f506f7ebed [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:35.386121 3436 raft_consensus.cc:2454] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P c5843bb7158442289c055d82b1b4110a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6e7d4bed9b44098a8a2a4692b745926 in term 1.
I20260808 23:31:35.386197 3806 raft_consensus.cc:2454] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P 2fa66641a264448f8f8e82f506f7ebed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6e7d4bed9b44098a8a2a4692b745926 in term 1.
I20260808 23:31:35.386469 3535 leader_election.cc:302] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6e7d4bed9b44098a8a2a4692b745926, c5843bb7158442289c055d82b1b4110a; no voters:
I20260808 23:31:35.386734 3910 raft_consensus.cc:2781] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:31:35.387148 3910 raft_consensus.cc:686] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926 [term 1 LEADER]: Becoming Leader. State: Replica: b6e7d4bed9b44098a8a2a4692b745926, State: Running, Role: LEADER
I20260808 23:31:35.387609 3910 consensus_queue.cc:237] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5843bb7158442289c055d82b1b4110a" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43299 } } peers { permanent_uuid: "b6e7d4bed9b44098a8a2a4692b745926" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 35559 } } peers { permanent_uuid: "2fa66641a264448f8f8e82f506f7ebed" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39999 } }
I20260808 23:31:35.391942 3219 catalog_manager.cc:5495] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926 reported cstate change: term changed from 0 to 1, leader changed from <none> to b6e7d4bed9b44098a8a2a4692b745926 (127.9.44.66). New cstate: current_term: 1 leader_uuid: "b6e7d4bed9b44098a8a2a4692b745926" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5843bb7158442289c055d82b1b4110a" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6e7d4bed9b44098a8a2a4692b745926" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 35559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fa66641a264448f8f8e82f506f7ebed" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39999 } health_report { overall_health: UNKNOWN } } }
I20260808 23:31:35.438265 3894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:35.432682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41938 (local address 127.9.44.66:35559)
0808 23:31:35.432884 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:35.432890 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:35.432970 (+ 80us) server_negotiation.cc:407] Connection header received
0808 23:31:35.433191 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:35.433196 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:35.433295 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:35.433421 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:31:35.433833 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.434627 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.436182 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.436591 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.436699 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:35.437187 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:31:35.437683 (+ 496us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:31:35.437820 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:35.437919 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 23:31:35.437999 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 23:31:35.446178 3806 raft_consensus.cc:1261] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P 2fa66641a264448f8f8e82f506f7ebed [term 1 FOLLOWER]: Refusing update from remote peer b6e7d4bed9b44098a8a2a4692b745926: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:31:35.446192 3436 raft_consensus.cc:1261] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P c5843bb7158442289c055d82b1b4110a [term 1 FOLLOWER]: Refusing update from remote peer b6e7d4bed9b44098a8a2a4692b745926: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:31:35.447091 3910 consensus_queue.cc:1035] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fa66641a264448f8f8e82f506f7ebed" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 39999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:31:35.447399 3916 consensus_queue.cc:1035] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5843bb7158442289c055d82b1b4110a" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 43299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:31:35.458037 3920 mvcc.cc:205] Tried to move back new op lower bound from 7316405843737513984 to 7316405843511414784. Current Snapshot: MvccSnapshot[applied={T|T < 7316405843737513984}]
I20260808 23:31:35.459556 3921 mvcc.cc:205] Tried to move back new op lower bound from 7316405843737513984 to 7316405843511414784. Current Snapshot: MvccSnapshot[applied={T|T < 7316405843737513984}]
I20260808 23:31:35.472998 3220 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:31:35.473266 3220 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:31:35.476938 3220 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 797b8309e0ff4cd19ab74a8c1e46e212: Sending DeleteTablet for 3 replicas of tablet 49b84ed65d0a43b9ad85a3b53a1e15f2
I20260808 23:31:35.479173 3410 tablet_service.cc:1514] Processing DeleteTablet for tablet 49b84ed65d0a43b9ad85a3b53a1e15f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:31:35 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:47174
I20260808 23:31:35.479780 3594 tablet_service.cc:1514] Processing DeleteTablet for tablet 49b84ed65d0a43b9ad85a3b53a1e15f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:31:35 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:41936
I20260808 23:31:35.479950 3933 tablet_replica.cc:330] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P c5843bb7158442289c055d82b1b4110a: stopping tablet replica
I20260808 23:31:35.480329 3933 raft_consensus.cc:2229] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P c5843bb7158442289c055d82b1b4110a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:31:35.480425 3775 tablet_service.cc:1514] Processing DeleteTablet for tablet 49b84ed65d0a43b9ad85a3b53a1e15f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:31:35 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:40946
I20260808 23:31:35.480789 3933 raft_consensus.cc:2258] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P c5843bb7158442289c055d82b1b4110a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:31:35.483755 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 3310
I20260808 23:31:35.486564 3936 tablet_replica.cc:330] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P 2fa66641a264448f8f8e82f506f7ebed: stopping tablet replica
I20260808 23:31:35.486934 3936 raft_consensus.cc:2229] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P 2fa66641a264448f8f8e82f506f7ebed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:31:35.487399 3936 raft_consensus.cc:2258] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P 2fa66641a264448f8f8e82f506f7ebed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:31:35.488147 3934 tablet_replica.cc:330] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926: stopping tablet replica
I20260808 23:31:35.488531 3934 raft_consensus.cc:2229] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:31:35.489156 3934 raft_consensus.cc:2258] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:31:35.491029 3933 ts_tablet_manager.cc:1905] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P c5843bb7158442289c055d82b1b4110a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:31:35.491182 3934 ts_tablet_manager.cc:1905] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:31:35.498029 3936 ts_tablet_manager.cc:1905] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P 2fa66641a264448f8f8e82f506f7ebed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:31:35.498448 3934 ts_tablet_manager.cc:1918] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:31:35.498615 3933 ts_tablet_manager.cc:1918] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P c5843bb7158442289c055d82b1b4110a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:31:35.498675 3934 log.cc:1196] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/49b84ed65d0a43b9ad85a3b53a1e15f2
I20260808 23:31:35.498765 3933 log.cc:1196] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P c5843bb7158442289c055d82b1b4110a: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-0/wal/wals/49b84ed65d0a43b9ad85a3b53a1e15f2
I20260808 23:31:35.499285 3933 ts_tablet_manager.cc:1939] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P c5843bb7158442289c055d82b1b4110a: Deleting consensus metadata
I20260808 23:31:35.499330 3934 ts_tablet_manager.cc:1939] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P b6e7d4bed9b44098a8a2a4692b745926: Deleting consensus metadata
I20260808 23:31:35.500994 3199 catalog_manager.cc:4841] TS b6e7d4bed9b44098a8a2a4692b745926 (127.9.44.66:35559): tablet 49b84ed65d0a43b9ad85a3b53a1e15f2 (table test-table [id=c871001570714995b82f3359c1aaf13c]) successfully deleted
I20260808 23:31:35.501302 3203 catalog_manager.cc:4841] TS c5843bb7158442289c055d82b1b4110a (127.9.44.65:43299): tablet 49b84ed65d0a43b9ad85a3b53a1e15f2 (table test-table [id=c871001570714995b82f3359c1aaf13c]) successfully deleted
Aug 08 23:31:35 dist-test-slave-6bbx krb5kdc[3156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.65@KRBTEST.COM
I20260808 23:31:35.507432 3936 ts_tablet_manager.cc:1918] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P 2fa66641a264448f8f8e82f506f7ebed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:31:35.507591 3936 log.cc:1196] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P 2fa66641a264448f8f8e82f506f7ebed: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/49b84ed65d0a43b9ad85a3b53a1e15f2
I20260808 23:31:35.508150 3936 ts_tablet_manager.cc:1939] T 49b84ed65d0a43b9ad85a3b53a1e15f2 P 2fa66641a264448f8f8e82f506f7ebed: Deleting consensus metadata
I20260808 23:31:35.509846 3202 catalog_manager.cc:4841] TS 2fa66641a264448f8f8e82f506f7ebed (127.9.44.67:39999): tablet 49b84ed65d0a43b9ad85a3b53a1e15f2 (table test-table [id=c871001570714995b82f3359c1aaf13c]) successfully deleted
I20260808 23:31:35.510649 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:35.484953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47176 (local address 127.9.44.65:43299)
0808 23:31:35.485134 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:35.485140 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:35.489857 (+ 4717us) server_negotiation.cc:407] Connection header received
0808 23:31:35.490134 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:35.490140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:35.490231 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:35.490304 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:35.492109 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.493285 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.495233 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.495938 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.496046 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:35.505179 (+ 9133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:35.505190 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:35.505203 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:35.505240 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:35.508028 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:35.508729 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:35.508735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:35.508738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:35.508870 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:35.509232 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:35.509236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:35.509238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:35.509590 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:35.509898 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:35.510204 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 23:31:35.510370 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:31:35.511327 3492 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:35.604876 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 3310
I20260808 23:31:35.631515 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 3507
Aug 08 23:31:35 dist-test-slave-6bbx krb5kdc[3156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.66@KRBTEST.COM
I20260808 23:31:35.649070 3894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:35.632291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41944 (local address 127.9.44.66:35559)
0808 23:31:35.632450 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:35.632455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:35.632665 (+ 210us) server_negotiation.cc:407] Connection header received
0808 23:31:35.632851 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:35.632855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:35.632956 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:35.633044 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:35.634634 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.635530 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.637589 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.638027 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.638118 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:35.643120 (+ 5002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:35.643131 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:35.643146 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:35.643185 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:35.646310 (+ 3125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:35.646949 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:35.646960 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:35.646962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:35.647107 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:35.647485 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:35.647489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:35.647491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:35.647903 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:35.648226 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:35.648515 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 23:31:35.648742 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 23:31:35.649734 3675 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:35.759850 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 3507
I20260808 23:31:35.785562 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 3685
Aug 08 23:31:35 dist-test-slave-6bbx krb5kdc[3156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.67@KRBTEST.COM
I20260808 23:31:35.799552 3896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:35.786314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40956 (local address 127.9.44.67:39999)
0808 23:31:35.786474 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:35.786478 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:35.786496 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:31:35.786649 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:35.786652 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:35.786733 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:35.786816 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:35.788067 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.788784 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.790067 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:35.790434 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:35.790519 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:35.794651 (+ 4132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:35.794665 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:35.794682 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:35.794731 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:35.797190 (+ 2459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:35.797777 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:35.797784 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:35.797786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:35.797920 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:35.798217 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:35.798220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:35.798222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:35.798554 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:35.798774 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:35.799001 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 23:31:35.799209 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 23:31:35.800197 3864 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:35.888859 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 3685
I20260808 23:31:35.913635 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 3175
I20260808 23:31:35.914795 3261 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:36.022161 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 3175
2026-08-08T23:31:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4446 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T23:31:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:31:36Z Disabled control of system clock
I20260808 23:31:36.106894 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:35105
--webserver_interface=127.9.44.126
--webserver_port=0
--builtin_ntp_servers=127.9.44.84:43905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:35105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 23:31:36.428005 3947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:36.428290 3947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:36.428385 3947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:36.435293 3947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:31:36.435389 3947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:36.435433 3947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:36.435487 3947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:31:36.435529 3947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:31:36.444222 3947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:43905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:35105
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:35105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:36.445353 3947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:36.447002 3947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:36.457870 3956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:36.457906 3959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:36.457962 3955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:36.458878 3947 server_base.cc:1034] running on GCE node
I20260808 23:31:36.459618 3947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:36.460944 3947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:36.462150 3947 hybrid_clock.cc:648] HybridClock initialized: now 1786231896462090 us; error 43 us; skew 500 ppm
I20260808 23:31:36.462481 3947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:31:36.468483 3947 webserver.cc:460] Webserver started at http://127.9.44.126:33715/ using document root <none> and password file <none>
I20260808 23:31:36.469131 3947 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:36.469234 3947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:36.469579 3947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:36.472504 3947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "8874b3ff80bc417b83b105fa12e5fad8"
format_stamp: "Formatted at 2026-08-08 23:31:36 on dist-test-slave-6bbx"
server_key: "3b4d4af045da8102ce43a3d91f4e5452"
server_key_iv: "70742df11e2ddc25e1002fb509f8cb82"
server_key_version: "encryptionkey@0"
I20260808 23:31:36.473321 3947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "8874b3ff80bc417b83b105fa12e5fad8"
format_stamp: "Formatted at 2026-08-08 23:31:36 on dist-test-slave-6bbx"
server_key: "3b4d4af045da8102ce43a3d91f4e5452"
server_key_iv: "70742df11e2ddc25e1002fb509f8cb82"
server_key_version: "encryptionkey@0"
I20260808 23:31:36.478381 3947 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:31:36.481735 3967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:36.483006 3947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:31:36.483193 3947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "8874b3ff80bc417b83b105fa12e5fad8"
format_stamp: "Formatted at 2026-08-08 23:31:36 on dist-test-slave-6bbx"
server_key: "3b4d4af045da8102ce43a3d91f4e5452"
server_key_iv: "70742df11e2ddc25e1002fb509f8cb82"
server_key_version: "encryptionkey@0"
I20260808 23:31:36.483385 3947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:36.510691 3947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:36.511435 3947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:36.511721 3947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:36.528899 3947 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:35105
I20260808 23:31:36.528932 4039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:35105 every 8 connection(s)
I20260808 23:31:36.530143 3947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:31:36.533646 4041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:36.538928 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 3947
I20260808 23:31:36.539119 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:31:36.539714 9393 external_mini_cluster.cc:1428] Setting key 116760da6ff0ab28e46989f335647e78
I20260808 23:31:36.544497 4041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8874b3ff80bc417b83b105fa12e5fad8: Bootstrap starting.
I20260808 23:31:36.547351 4041 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8874b3ff80bc417b83b105fa12e5fad8: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:36.548316 4041 log.cc:824] T 00000000000000000000000000000000 P 8874b3ff80bc417b83b105fa12e5fad8: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:36.551298 4041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8874b3ff80bc417b83b105fa12e5fad8: No bootstrap required, opened a new log
I20260808 23:31:36.552810 4044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:36.541663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40904 (local address 127.9.44.126:35105)
0808 23:31:36.542692 (+ 1029us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:36.542703 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:36.542741 (+ 38us) server_negotiation.cc:407] Connection header received
0808 23:31:36.544420 (+ 1679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:36.544442 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:36.544888 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:36.545219 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:36.545753 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:36.546999 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:36.548427 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:36.549400 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:36.549587 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:36.550722 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:36.550730 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:36.550743 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:36.551225 (+ 482us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:36.551301 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:36.551325 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:36.551424 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:36.551683 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 23:31:36.551951 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":377,"threads_started":1}
I20260808 23:31:36.558934 4041 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8874b3ff80bc417b83b105fa12e5fad8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8874b3ff80bc417b83b105fa12e5fad8" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35105 } }
I20260808 23:31:36.559252 4041 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8874b3ff80bc417b83b105fa12e5fad8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:36.559336 4041 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8874b3ff80bc417b83b105fa12e5fad8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8874b3ff80bc417b83b105fa12e5fad8, State: Initialized, Role: FOLLOWER
I20260808 23:31:36.559877 4041 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8874b3ff80bc417b83b105fa12e5fad8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8874b3ff80bc417b83b105fa12e5fad8" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35105 } }
I20260808 23:31:36.560099 4041 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8874b3ff80bc417b83b105fa12e5fad8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:31:36.560226 4041 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8874b3ff80bc417b83b105fa12e5fad8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:31:36.560375 4041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8874b3ff80bc417b83b105fa12e5fad8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:36.562575 4041 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8874b3ff80bc417b83b105fa12e5fad8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8874b3ff80bc417b83b105fa12e5fad8" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35105 } }
I20260808 23:31:36.563076 4041 leader_election.cc:302] T 00000000000000000000000000000000 P 8874b3ff80bc417b83b105fa12e5fad8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8874b3ff80bc417b83b105fa12e5fad8; no voters:
I20260808 23:31:36.563571 4041 leader_election.cc:288] T 00000000000000000000000000000000 P 8874b3ff80bc417b83b105fa12e5fad8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:31:36.563699 4047 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8874b3ff80bc417b83b105fa12e5fad8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:31:36.564368 4047 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8874b3ff80bc417b83b105fa12e5fad8 [term 1 LEADER]: Becoming Leader. State: Replica: 8874b3ff80bc417b83b105fa12e5fad8, State: Running, Role: LEADER
I20260808 23:31:36.564878 4047 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8874b3ff80bc417b83b105fa12e5fad8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8874b3ff80bc417b83b105fa12e5fad8" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35105 } }
I20260808 23:31:36.565445 4041 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8874b3ff80bc417b83b105fa12e5fad8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:31:36.567597 4049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8874b3ff80bc417b83b105fa12e5fad8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8874b3ff80bc417b83b105fa12e5fad8. Latest consensus state: current_term: 1 leader_uuid: "8874b3ff80bc417b83b105fa12e5fad8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8874b3ff80bc417b83b105fa12e5fad8" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35105 } } }
I20260808 23:31:36.568382 4049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8874b3ff80bc417b83b105fa12e5fad8 [sys.catalog]: This master's current role is: LEADER
I20260808 23:31:36.567807 4048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8874b3ff80bc417b83b105fa12e5fad8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8874b3ff80bc417b83b105fa12e5fad8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8874b3ff80bc417b83b105fa12e5fad8" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 35105 } } }
I20260808 23:31:36.568760 4048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8874b3ff80bc417b83b105fa12e5fad8 [sys.catalog]: This master's current role is: LEADER
I20260808 23:31:36.572433 4057 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:31:36.577404 4057 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:31:36.585803 4057 catalog_manager.cc:1347] Generated new cluster ID: d652575ccc59443e8d5185498042e371
I20260808 23:31:36.585908 4057 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:31:36.626302 4057 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:31:36.627494 4057 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:31:36.642334 4057 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8874b3ff80bc417b83b105fa12e5fad8: Generated new TSK 0
I20260808 23:31:36.643078 4057 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:31:36.651262 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:0
--local_ip_for_outbound_sockets=127.9.44.65
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:35105
--builtin_ntp_servers=127.9.44.84:43905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 23:31:36.983695 4072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:36.983937 4072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:36.984057 4072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:36.991031 4072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:36.991122 4072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:36.991277 4072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:31:36.999950 4072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:43905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:35105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:37.001048 4072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:37.002730 4072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:37.014855 4081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:37.014889 4079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:37.015154 4078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:37.015683 4072 server_base.cc:1034] running on GCE node
I20260808 23:31:37.016229 4072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:37.017058 4072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:37.018257 4072 hybrid_clock.cc:648] HybridClock initialized: now 1786231897018211 us; error 40 us; skew 500 ppm
I20260808 23:31:37.018548 4072 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:31:37.020675 4072 webserver.cc:460] Webserver started at http://127.9.44.65:41077/ using document root <none> and password file <none>
I20260808 23:31:37.021291 4072 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:37.021401 4072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:37.021697 4072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:37.024410 4072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-0/data/instance:
uuid: "d5479a16d3194828aee855b8dc1eb277"
format_stamp: "Formatted at 2026-08-08 23:31:37 on dist-test-slave-6bbx"
server_key: "76f9c314fd2cb5e9eaa93009d16989e5"
server_key_iv: "80264cd3ae509e7d159f8cfbfd2922bb"
server_key_version: "encryptionkey@0"
I20260808 23:31:37.025228 4072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance:
uuid: "d5479a16d3194828aee855b8dc1eb277"
format_stamp: "Formatted at 2026-08-08 23:31:37 on dist-test-slave-6bbx"
server_key: "76f9c314fd2cb5e9eaa93009d16989e5"
server_key_iv: "80264cd3ae509e7d159f8cfbfd2922bb"
server_key_version: "encryptionkey@0"
I20260808 23:31:37.029940 4072 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 23:31:37.033224 4093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:37.034407 4072 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 23:31:37.034556 4072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "d5479a16d3194828aee855b8dc1eb277"
format_stamp: "Formatted at 2026-08-08 23:31:37 on dist-test-slave-6bbx"
server_key: "76f9c314fd2cb5e9eaa93009d16989e5"
server_key_iv: "80264cd3ae509e7d159f8cfbfd2922bb"
server_key_version: "encryptionkey@0"
I20260808 23:31:37.034747 4072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:37.050634 4072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:37.051342 4072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:37.051599 4072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:37.052539 4072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:31:37.054013 4072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:31:37.054116 4072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:37.054219 4072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:31:37.054291 4072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:37.089048 4072 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:33181
I20260808 23:31:37.089098 4249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:33181 every 8 connection(s)
I20260808 23:31:37.090265 4072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:31:37.091228 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 4072
I20260808 23:31:37.091400 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance
I20260808 23:31:37.091892 9393 external_mini_cluster.cc:1428] Setting key 5cd3e93ed7069fc3c0831a23fb43a3cf
I20260808 23:31:37.095145 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:0
--local_ip_for_outbound_sockets=127.9.44.66
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:35105
--builtin_ntp_servers=127.9.44.84:43905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 23:31:37.105333 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:37.092787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:35267 (local address 127.9.44.126:35105)
0808 23:31:37.094987 (+ 2200us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:37.094993 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:37.096385 (+ 1392us) server_negotiation.cc:407] Connection header received
0808 23:31:37.098586 (+ 2201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:37.098590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:37.098655 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:37.098746 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:37.100036 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:37.100769 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:37.102777 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:37.103146 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:37.103244 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:37.104650 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:37.104655 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:37.104659 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:37.104675 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:37.104720 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:37.104732 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:37.104798 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:37.105007 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 23:31:37.105082 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2069,"thread_start_us":1967,"threads_started":1}
I20260808 23:31:37.106529 4256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:37.093332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35105 (local address 127.9.44.65:35267)
0808 23:31:37.096238 (+ 2906us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:37.096284 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:37.098321 (+ 2037us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:37.098993 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:37.099003 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:37.099106 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:31:37.099884 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:37.099908 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:37.100910 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:37.100915 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:37.102656 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:37.102673 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:37.103352 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:37.103367 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:37.103632 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:37.104331 (+ 699us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:31:37.104335 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:31:37.104392 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:31:37.104479 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:31:37.104817 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:31:37.104826 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:31:37.104833 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 23:31:37.104992 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 23:31:37.105344 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2407,"thread_start_us":534,"threads_started":1}
I20260808 23:31:37.107960 4250 heartbeater.cc:347] Connected to a master server at 127.9.44.126:35105
I20260808 23:31:37.108234 4250 heartbeater.cc:464] Registering TS with master...
I20260808 23:31:37.108917 4250 heartbeater.cc:511] Master 127.9.44.126:35105 requested a full tablet report, sending...
I20260808 23:31:37.110777 3986 ts_manager.cc:194] Registered new tserver with Master: d5479a16d3194828aee855b8dc1eb277 (127.9.44.65:33181)
I20260808 23:31:37.112250 3986 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.44.65:35267
W20260808 23:31:37.420503 4258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:37.420782 4258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:37.420909 4258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:37.428293 4258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:37.428398 4258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:37.428544 4258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:31:37.437224 4258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:43905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:35105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:37.438445 4258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:37.440204 4258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:37.452488 4268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:37.452557 4264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:37.452534 4265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:37.453477 4258 server_base.cc:1034] running on GCE node
I20260808 23:31:37.453991 4258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:37.454892 4258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:37.456089 4258 hybrid_clock.cc:648] HybridClock initialized: now 1786231897456035 us; error 40 us; skew 500 ppm
I20260808 23:31:37.456386 4258 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:31:37.458415 4258 webserver.cc:460] Webserver started at http://127.9.44.66:38283/ using document root <none> and password file <none>
I20260808 23:31:37.459046 4258 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:37.459148 4258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:37.459448 4258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:37.462186 4258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-1/data/instance:
uuid: "ca00680d2a1f4210940ca453bd6bff68"
format_stamp: "Formatted at 2026-08-08 23:31:37 on dist-test-slave-6bbx"
server_key: "58bc0c83f74729b6d0d88062faee7b85"
server_key_iv: "18db5c785ac0b6bce641c81c66f05cf1"
server_key_version: "encryptionkey@0"
I20260808 23:31:37.462934 4258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance:
uuid: "ca00680d2a1f4210940ca453bd6bff68"
format_stamp: "Formatted at 2026-08-08 23:31:37 on dist-test-slave-6bbx"
server_key: "58bc0c83f74729b6d0d88062faee7b85"
server_key_iv: "18db5c785ac0b6bce641c81c66f05cf1"
server_key_version: "encryptionkey@0"
I20260808 23:31:37.467905 4258 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260808 23:31:37.471033 4276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:37.472255 4258 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 23:31:37.472424 4258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "ca00680d2a1f4210940ca453bd6bff68"
format_stamp: "Formatted at 2026-08-08 23:31:37 on dist-test-slave-6bbx"
server_key: "58bc0c83f74729b6d0d88062faee7b85"
server_key_iv: "18db5c785ac0b6bce641c81c66f05cf1"
server_key_version: "encryptionkey@0"
I20260808 23:31:37.472626 4258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:37.497941 4258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:37.498646 4258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:37.498898 4258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:37.499802 4258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:31:37.501189 4258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:31:37.501286 4258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:37.501395 4258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:31:37.501471 4258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:37.531021 4258 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:46027
I20260808 23:31:37.531092 4430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:46027 every 8 connection(s)
I20260808 23:31:37.532224 4258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:31:37.535439 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 4258
I20260808 23:31:37.535621 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance
I20260808 23:31:37.536074 9393 external_mini_cluster.cc:1428] Setting key 729626a9dd6d039cfaf2aa48d0c451af
I20260808 23:31:37.538806 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:0
--local_ip_for_outbound_sockets=127.9.44.67
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:35105
--builtin_ntp_servers=127.9.44.84:43905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 23:31:37.557044 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:37.534633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:58185 (local address 127.9.44.126:35105)
0808 23:31:37.534794 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:37.534799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:37.548653 (+ 13854us) server_negotiation.cc:407] Connection header received
0808 23:31:37.550821 (+ 2168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:37.550826 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:37.550906 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:37.551013 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:37.552047 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:37.552766 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:37.554224 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:37.554626 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:37.554737 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:37.556276 (+ 1539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:37.556283 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:37.556288 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:37.556306 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:37.556337 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:37.556349 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:37.556432 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:37.556657 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 23:31:37.556756 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 23:31:37.557973 4436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:37.535202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35105 (local address 127.9.44.66:58185)
0808 23:31:37.548485 (+ 13283us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:37.548532 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:37.550539 (+ 2007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:37.551190 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:37.551198 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:37.551288 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:31:37.551884 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:37.551903 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:37.552885 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:37.552888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:37.554077 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:37.554090 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:37.554866 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:37.554876 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:37.555175 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:37.555976 (+ 801us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:31:37.555981 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:31:37.556022 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:31:37.556101 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:31:37.556445 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:31:37.556457 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:31:37.556465 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 23:31:37.556615 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 23:31:37.556993 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12741,"thread_start_us":12517,"threads_started":1}
I20260808 23:31:37.559306 4434 heartbeater.cc:347] Connected to a master server at 127.9.44.126:35105
I20260808 23:31:37.559588 4434 heartbeater.cc:464] Registering TS with master...
I20260808 23:31:37.560297 4434 heartbeater.cc:511] Master 127.9.44.126:35105 requested a full tablet report, sending...
I20260808 23:31:37.561623 3986 ts_manager.cc:194] Registered new tserver with Master: ca00680d2a1f4210940ca453bd6bff68 (127.9.44.66:46027)
I20260808 23:31:37.562387 3986 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.44.66:58185
W20260808 23:31:37.872205 4439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:37.872478 4439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:37.872613 4439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:37.880816 4439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:37.880913 4439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:37.881068 4439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:31:37.890149 4439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:43905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:35105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:37.891355 4439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:37.893128 4439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:37.905504 4446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:37.905542 4452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:37.905463 4449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:37.906468 4439 server_base.cc:1034] running on GCE node
I20260808 23:31:37.907011 4439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:37.907881 4439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:37.909119 4439 hybrid_clock.cc:648] HybridClock initialized: now 1786231897909054 us; error 49 us; skew 500 ppm
I20260808 23:31:37.909441 4439 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:31:37.911597 4439 webserver.cc:460] Webserver started at http://127.9.44.67:45883/ using document root <none> and password file <none>
I20260808 23:31:37.912274 4439 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:37.912362 4439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:37.912626 4439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:37.915328 4439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-2/data/instance:
uuid: "8a07026e5cbb4bf59402cf9c19918ab1"
format_stamp: "Formatted at 2026-08-08 23:31:37 on dist-test-slave-6bbx"
server_key: "ec2a18597c0950ddeb32c0cd34621224"
server_key_iv: "7e51d339355b8f79fccae52845bc753b"
server_key_version: "encryptionkey@0"
I20260808 23:31:37.916219 4439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance:
uuid: "8a07026e5cbb4bf59402cf9c19918ab1"
format_stamp: "Formatted at 2026-08-08 23:31:37 on dist-test-slave-6bbx"
server_key: "ec2a18597c0950ddeb32c0cd34621224"
server_key_iv: "7e51d339355b8f79fccae52845bc753b"
server_key_version: "encryptionkey@0"
I20260808 23:31:37.921535 4439 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 23:31:37.924865 4460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:37.926074 4439 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:31:37.926229 4439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "8a07026e5cbb4bf59402cf9c19918ab1"
format_stamp: "Formatted at 2026-08-08 23:31:37 on dist-test-slave-6bbx"
server_key: "ec2a18597c0950ddeb32c0cd34621224"
server_key_iv: "7e51d339355b8f79fccae52845bc753b"
server_key_version: "encryptionkey@0"
I20260808 23:31:37.926426 4439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:37.943313 4439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:37.944063 4439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:37.944314 4439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:37.945192 4439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:31:37.946555 4439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:31:37.946656 4439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:37.946763 4439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:31:37.946835 4439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:37.976824 4439 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:38691
I20260808 23:31:37.976897 4629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:38691 every 8 connection(s)
I20260808 23:31:37.978106 4439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:31:37.979413 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 4439
I20260808 23:31:37.979581 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance
I20260808 23:31:37.980134 9393 external_mini_cluster.cc:1428] Setting key c600327356237af7c118eae71e48380e
I20260808 23:31:37.990496 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:37.980574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:32777 (local address 127.9.44.126:35105)
0808 23:31:37.980792 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:37.980799 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:37.982352 (+ 1553us) server_negotiation.cc:407] Connection header received
0808 23:31:37.984282 (+ 1930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:37.984288 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:37.984371 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:37.984459 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:37.985495 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:37.986255 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:37.987899 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:37.988375 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:37.988493 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:37.989691 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:37.989697 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:37.989701 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:37.989718 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:37.989749 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:37.989761 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:37.989833 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:37.990040 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 23:31:37.990146 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 23:31:37.991379 4634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:37.981150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:35105 (local address 127.9.44.67:32777)
0808 23:31:37.982223 (+ 1073us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:37.982259 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:37.984033 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:37.984709 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:37.984716 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:37.984798 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:31:37.985333 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:37.985351 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:37.986397 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:37.986402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:37.987769 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:37.987783 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:37.988594 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:37.988602 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:37.988819 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:37.989397 (+ 578us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:31:37.989399 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:31:37.989435 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:31:37.989508 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:31:37.989848 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:31:37.989860 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:31:37.989867 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 23:31:37.990015 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 23:31:37.990369 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":320,"threads_started":1}
I20260808 23:31:37.992615 4630 heartbeater.cc:347] Connected to a master server at 127.9.44.126:35105
I20260808 23:31:37.992875 4630 heartbeater.cc:464] Registering TS with master...
I20260808 23:31:37.993640 4630 heartbeater.cc:511] Master 127.9.44.126:35105 requested a full tablet report, sending...
I20260808 23:31:37.995055 3986 ts_manager.cc:194] Registered new tserver with Master: 8a07026e5cbb4bf59402cf9c19918ab1 (127.9.44.67:38691)
I20260808 23:31:37.995893 3986 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.44.67:32777
I20260808 23:31:38.006770 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:31:38.016182 4255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:38.010500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50156 (local address 127.9.44.126:35105)
0808 23:31:38.010681 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:38.010688 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:38.011098 (+ 410us) server_negotiation.cc:407] Connection header received
0808 23:31:38.011278 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:38.011284 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:38.011355 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:38.011434 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:38.011952 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.012614 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.014042 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.014446 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.014550 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:38.015534 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:38.015539 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:38.015542 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:38.015558 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:38.015588 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:38.015600 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:38.015700 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:38.015834 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 23:31:38.015922 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 23:31:38.019347 3986 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:31:38.025924 3986 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:31:38.046988 4655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:38.039108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:46027 (local address 127.0.0.1:38508)
0808 23:31:38.039623 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:38.039654 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:38.039776 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:38.041169 (+ 1393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:38.041174 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:38.041199 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:38.041521 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:38.041532 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.042928 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.042932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:38.044783 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:38.044807 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.046084 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.046091 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:38.046287 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:38.046337 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 23:31:38.046447 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 23:31:38.046633 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":2,"thread_start_us":834,"threads_started":1}
I20260808 23:31:38.047298 4646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:38.036796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:33181 (local address 127.0.0.1:53576)
0808 23:31:38.037557 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:38.037602 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:38.037776 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:38.038807 (+ 1031us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:38.038815 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:38.038850 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:38.039264 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:38.039278 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.042082 (+ 2804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.042087 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:38.044642 (+ 2555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:38.044657 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.046482 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.046489 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:38.046681 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:38.046712 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 23:31:38.046808 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 23:31:38.046919 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":106,"thread_start_us":298,"threads_started":1}
I20260808 23:31:38.047891 4650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:38.036978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53576 (local address 127.9.44.65:33181)
0808 23:31:38.038010 (+ 1032us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:38.038016 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:38.038038 (+ 22us) server_negotiation.cc:407] Connection header received
0808 23:31:38.038123 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:38.038141 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:38.038370 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:38.038567 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:38.040600 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.041955 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.044871 (+ 2916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.046286 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.046426 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:38.046659 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:38.046861 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 23:31:38.047026 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":881,"thread_start_us":759,"threads_started":1}
I20260808 23:31:38.047602 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:38.038059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:38691 (local address 127.0.0.1:49212)
0808 23:31:38.038639 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:38.038663 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:38.038779 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:38.039917 (+ 1138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:38.039922 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:38.039944 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:38.040389 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:38.040402 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.041969 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.041975 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:38.044871 (+ 2896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:38.044883 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.046800 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.046807 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:38.046981 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:38.047013 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 23:31:38.047095 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 23:31:38.047231 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":72,"thread_start_us":261,"threads_started":1}
I20260808 23:31:38.048395 4654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:38.038332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49212 (local address 127.9.44.67:38691)
0808 23:31:38.039374 (+ 1042us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:38.039382 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:38.039414 (+ 32us) server_negotiation.cc:407] Connection header received
0808 23:31:38.039503 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:38.039509 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:38.039761 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:38.039941 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:38.040592 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.041857 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.045062 (+ 3205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.046348 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.046459 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:38.046681 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:38.047392 (+ 711us) server_negotiation.cc:299] Negotiation successful
0808 23:31:38.047559 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":887,"thread_start_us":739,"threads_started":1}
I20260808 23:31:38.048553 4656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:38.039752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38508 (local address 127.9.44.66:46027)
0808 23:31:38.040613 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:38.040620 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:38.040648 (+ 28us) server_negotiation.cc:407] Connection header received
0808 23:31:38.040751 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:38.040758 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:38.040968 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:38.041149 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:38.041686 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.042794 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.044999 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.045906 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.046032 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:38.046303 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:38.047394 (+ 1091us) server_negotiation.cc:299] Negotiation successful
0808 23:31:38.047571 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":594,"threads_started":1}
I20260808 23:31:38.051489 4153 tablet_service.cc:1467] Processing CreateTablet for tablet 9eed985f4fda4d1985ad52bc437dcc60 (DEFAULT_TABLE table=test-table [id=bc8fe013ce66498d915d65d089895875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:31:38.051872 4337 tablet_service.cc:1467] Processing CreateTablet for tablet 9eed985f4fda4d1985ad52bc437dcc60 (DEFAULT_TABLE table=test-table [id=bc8fe013ce66498d915d65d089895875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:31:38.051872 4530 tablet_service.cc:1467] Processing CreateTablet for tablet 9eed985f4fda4d1985ad52bc437dcc60 (DEFAULT_TABLE table=test-table [id=bc8fe013ce66498d915d65d089895875]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:31:38.052866 4153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eed985f4fda4d1985ad52bc437dcc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:38.053248 4337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eed985f4fda4d1985ad52bc437dcc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:38.053248 4530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eed985f4fda4d1985ad52bc437dcc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:38.063510 4657 tablet_bootstrap.cc:492] T 9eed985f4fda4d1985ad52bc437dcc60 P d5479a16d3194828aee855b8dc1eb277: Bootstrap starting.
I20260808 23:31:38.067096 4659 tablet_bootstrap.cc:492] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68: Bootstrap starting.
I20260808 23:31:38.067287 4657 tablet_bootstrap.cc:654] T 9eed985f4fda4d1985ad52bc437dcc60 P d5479a16d3194828aee855b8dc1eb277: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:38.068631 4660 tablet_bootstrap.cc:492] T 9eed985f4fda4d1985ad52bc437dcc60 P 8a07026e5cbb4bf59402cf9c19918ab1: Bootstrap starting.
I20260808 23:31:38.068634 4657 log.cc:824] T 9eed985f4fda4d1985ad52bc437dcc60 P d5479a16d3194828aee855b8dc1eb277: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:38.070649 4659 tablet_bootstrap.cc:654] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:38.071589 4660 tablet_bootstrap.cc:654] T 9eed985f4fda4d1985ad52bc437dcc60 P 8a07026e5cbb4bf59402cf9c19918ab1: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:38.071800 4657 tablet_bootstrap.cc:492] T 9eed985f4fda4d1985ad52bc437dcc60 P d5479a16d3194828aee855b8dc1eb277: No bootstrap required, opened a new log
I20260808 23:31:38.071926 4659 log.cc:824] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:38.072086 4657 ts_tablet_manager.cc:1397] T 9eed985f4fda4d1985ad52bc437dcc60 P d5479a16d3194828aee855b8dc1eb277: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260808 23:31:38.072732 4660 log.cc:824] T 9eed985f4fda4d1985ad52bc437dcc60 P 8a07026e5cbb4bf59402cf9c19918ab1: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:38.074903 4659 tablet_bootstrap.cc:492] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68: No bootstrap required, opened a new log
I20260808 23:31:38.075160 4659 ts_tablet_manager.cc:1397] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 23:31:38.081727 4660 tablet_bootstrap.cc:492] T 9eed985f4fda4d1985ad52bc437dcc60 P 8a07026e5cbb4bf59402cf9c19918ab1: No bootstrap required, opened a new log
I20260808 23:31:38.081957 4660 ts_tablet_manager.cc:1397] T 9eed985f4fda4d1985ad52bc437dcc60 P 8a07026e5cbb4bf59402cf9c19918ab1: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.004s
I20260808 23:31:38.082371 4657 raft_consensus.cc:348] T 9eed985f4fda4d1985ad52bc437dcc60 P d5479a16d3194828aee855b8dc1eb277 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5479a16d3194828aee855b8dc1eb277" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 33181 } } peers { permanent_uuid: "8a07026e5cbb4bf59402cf9c19918ab1" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38691 } } peers { permanent_uuid: "ca00680d2a1f4210940ca453bd6bff68" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 46027 } }
I20260808 23:31:38.082799 4657 raft_consensus.cc:374] T 9eed985f4fda4d1985ad52bc437dcc60 P d5479a16d3194828aee855b8dc1eb277 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:38.082892 4657 raft_consensus.cc:729] T 9eed985f4fda4d1985ad52bc437dcc60 P d5479a16d3194828aee855b8dc1eb277 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5479a16d3194828aee855b8dc1eb277, State: Initialized, Role: FOLLOWER
I20260808 23:31:38.083468 4657 consensus_queue.cc:260] T 9eed985f4fda4d1985ad52bc437dcc60 P d5479a16d3194828aee855b8dc1eb277 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5479a16d3194828aee855b8dc1eb277" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 33181 } } peers { permanent_uuid: "8a07026e5cbb4bf59402cf9c19918ab1" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38691 } } peers { permanent_uuid: "ca00680d2a1f4210940ca453bd6bff68" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 46027 } }
I20260808 23:31:38.084589 4250 heartbeater.cc:503] Master 127.9.44.126:35105 was elected leader, sending a full tablet report...
I20260808 23:31:38.085188 4657 ts_tablet_manager.cc:1428] T 9eed985f4fda4d1985ad52bc437dcc60 P d5479a16d3194828aee855b8dc1eb277: Time spent starting tablet: real 0.013s user 0.005s sys 0.007s
I20260808 23:31:38.086334 4659 raft_consensus.cc:348] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5479a16d3194828aee855b8dc1eb277" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 33181 } } peers { permanent_uuid: "8a07026e5cbb4bf59402cf9c19918ab1" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38691 } } peers { permanent_uuid: "ca00680d2a1f4210940ca453bd6bff68" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 46027 } }
I20260808 23:31:38.086674 4659 raft_consensus.cc:374] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:38.086786 4659 raft_consensus.cc:729] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca00680d2a1f4210940ca453bd6bff68, State: Initialized, Role: FOLLOWER
I20260808 23:31:38.087357 4659 consensus_queue.cc:260] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5479a16d3194828aee855b8dc1eb277" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 33181 } } peers { permanent_uuid: "8a07026e5cbb4bf59402cf9c19918ab1" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38691 } } peers { permanent_uuid: "ca00680d2a1f4210940ca453bd6bff68" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 46027 } }
I20260808 23:31:38.088402 4434 heartbeater.cc:503] Master 127.9.44.126:35105 was elected leader, sending a full tablet report...
I20260808 23:31:38.088985 4659 ts_tablet_manager.cc:1428] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68: Time spent starting tablet: real 0.014s user 0.009s sys 0.003s
I20260808 23:31:38.090451 4660 raft_consensus.cc:348] T 9eed985f4fda4d1985ad52bc437dcc60 P 8a07026e5cbb4bf59402cf9c19918ab1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5479a16d3194828aee855b8dc1eb277" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 33181 } } peers { permanent_uuid: "8a07026e5cbb4bf59402cf9c19918ab1" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38691 } } peers { permanent_uuid: "ca00680d2a1f4210940ca453bd6bff68" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 46027 } }
I20260808 23:31:38.090922 4660 raft_consensus.cc:374] T 9eed985f4fda4d1985ad52bc437dcc60 P 8a07026e5cbb4bf59402cf9c19918ab1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:38.091012 4660 raft_consensus.cc:729] T 9eed985f4fda4d1985ad52bc437dcc60 P 8a07026e5cbb4bf59402cf9c19918ab1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a07026e5cbb4bf59402cf9c19918ab1, State: Initialized, Role: FOLLOWER
I20260808 23:31:38.091532 4660 consensus_queue.cc:260] T 9eed985f4fda4d1985ad52bc437dcc60 P 8a07026e5cbb4bf59402cf9c19918ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5479a16d3194828aee855b8dc1eb277" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 33181 } } peers { permanent_uuid: "8a07026e5cbb4bf59402cf9c19918ab1" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38691 } } peers { permanent_uuid: "ca00680d2a1f4210940ca453bd6bff68" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 46027 } }
W20260808 23:31:38.095892 4251 tablet.cc:2369] T 9eed985f4fda4d1985ad52bc437dcc60 P d5479a16d3194828aee855b8dc1eb277: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:31:38.098489 4630 heartbeater.cc:503] Master 127.9.44.126:35105 was elected leader, sending a full tablet report...
I20260808 23:31:38.099040 4660 ts_tablet_manager.cc:1428] T 9eed985f4fda4d1985ad52bc437dcc60 P 8a07026e5cbb4bf59402cf9c19918ab1: Time spent starting tablet: real 0.017s user 0.009s sys 0.008s
I20260808 23:31:38.115538 4666 raft_consensus.cc:482] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:31:38.115820 4666 raft_consensus.cc:504] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5479a16d3194828aee855b8dc1eb277" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 33181 } } peers { permanent_uuid: "8a07026e5cbb4bf59402cf9c19918ab1" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38691 } } peers { permanent_uuid: "ca00680d2a1f4210940ca453bd6bff68" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 46027 } }
I20260808 23:31:38.117277 4666 leader_election.cc:288] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5479a16d3194828aee855b8dc1eb277 (127.9.44.65:33181), 8a07026e5cbb4bf59402cf9c19918ab1 (127.9.44.67:38691)
I20260808 23:31:38.125526 4650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:38.117599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:46361 (local address 127.9.44.65:33181)
0808 23:31:38.117896 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:38.117903 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:38.118566 (+ 663us) server_negotiation.cc:407] Connection header received
0808 23:31:38.118805 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:38.118810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:38.118889 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:38.118960 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:38.119581 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.120586 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.123340 (+ 2754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.124538 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.124673 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:38.124759 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:38.125160 (+ 401us) server_negotiation.cc:299] Negotiation successful
0808 23:31:38.125285 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260808 23:31:38.125515 4654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:38.117662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:56845 (local address 127.9.44.67:38691)
0808 23:31:38.118040 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:38.118046 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:38.118271 (+ 225us) server_negotiation.cc:407] Connection header received
0808 23:31:38.118593 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:38.118598 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:38.118666 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:38.118758 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:38.119404 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.120528 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.123351 (+ 2823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.124563 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.124673 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:38.124766 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:38.125158 (+ 392us) server_negotiation.cc:299] Negotiation successful
0808 23:31:38.125285 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259}
I20260808 23:31:38.125689 4669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:38.117500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:33181 (local address 127.9.44.66:46361)
0808 23:31:38.118476 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:38.118490 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:38.118657 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:38.119011 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:38.119015 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:38.119036 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:38.119450 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:38.119459 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.120695 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.120700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:38.123133 (+ 2433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:38.123148 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.124759 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.124766 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:38.124975 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:38.125034 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 23:31:38.125116 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 23:31:38.125286 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"mutex_wait_us":180,"thread_start_us":303,"threads_started":1}
I20260808 23:31:38.125711 4670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:38.117489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:38691 (local address 127.9.44.66:56845)
0808 23:31:38.118165 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:38.118187 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:38.118421 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:38.118816 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:38.118820 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:38.118844 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:38.119258 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:38.119269 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.120673 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.120678 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:38.123133 (+ 2455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:38.123148 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.124746 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.124753 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:38.124980 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:38.125028 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 23:31:38.125120 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 23:31:38.125286 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"mutex_wait_us":169,"thread_start_us":741,"threads_started":1}
I20260808 23:31:38.126748 4182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eed985f4fda4d1985ad52bc437dcc60" candidate_uuid: "ca00680d2a1f4210940ca453bd6bff68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5479a16d3194828aee855b8dc1eb277" is_pre_election: true
I20260808 23:31:38.126749 4558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eed985f4fda4d1985ad52bc437dcc60" candidate_uuid: "ca00680d2a1f4210940ca453bd6bff68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a07026e5cbb4bf59402cf9c19918ab1" is_pre_election: true
I20260808 23:31:38.127274 4182 raft_consensus.cc:2454] T 9eed985f4fda4d1985ad52bc437dcc60 P d5479a16d3194828aee855b8dc1eb277 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca00680d2a1f4210940ca453bd6bff68 in term 0.
I20260808 23:31:38.127274 4558 raft_consensus.cc:2454] T 9eed985f4fda4d1985ad52bc437dcc60 P 8a07026e5cbb4bf59402cf9c19918ab1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca00680d2a1f4210940ca453bd6bff68 in term 0.
I20260808 23:31:38.127915 4277 leader_election.cc:302] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a07026e5cbb4bf59402cf9c19918ab1, ca00680d2a1f4210940ca453bd6bff68; no voters:
I20260808 23:31:38.128232 4666 raft_consensus.cc:2781] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:31:38.128386 4666 raft_consensus.cc:482] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:31:38.128495 4666 raft_consensus.cc:3037] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:38.130700 4666 raft_consensus.cc:504] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5479a16d3194828aee855b8dc1eb277" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 33181 } } peers { permanent_uuid: "8a07026e5cbb4bf59402cf9c19918ab1" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38691 } } peers { permanent_uuid: "ca00680d2a1f4210940ca453bd6bff68" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 46027 } }
I20260808 23:31:38.131269 4666 leader_election.cc:288] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68 [CANDIDATE]: Term 1 election: Requested vote from peers d5479a16d3194828aee855b8dc1eb277 (127.9.44.65:33181), 8a07026e5cbb4bf59402cf9c19918ab1 (127.9.44.67:38691)
I20260808 23:31:38.131606 4182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eed985f4fda4d1985ad52bc437dcc60" candidate_uuid: "ca00680d2a1f4210940ca453bd6bff68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5479a16d3194828aee855b8dc1eb277"
I20260808 23:31:38.131748 4558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eed985f4fda4d1985ad52bc437dcc60" candidate_uuid: "ca00680d2a1f4210940ca453bd6bff68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a07026e5cbb4bf59402cf9c19918ab1"
I20260808 23:31:38.131830 4182 raft_consensus.cc:3037] T 9eed985f4fda4d1985ad52bc437dcc60 P d5479a16d3194828aee855b8dc1eb277 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:38.131946 4558 raft_consensus.cc:3037] T 9eed985f4fda4d1985ad52bc437dcc60 P 8a07026e5cbb4bf59402cf9c19918ab1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:38.134936 4558 raft_consensus.cc:2454] T 9eed985f4fda4d1985ad52bc437dcc60 P 8a07026e5cbb4bf59402cf9c19918ab1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca00680d2a1f4210940ca453bd6bff68 in term 1.
I20260808 23:31:38.134935 4182 raft_consensus.cc:2454] T 9eed985f4fda4d1985ad52bc437dcc60 P d5479a16d3194828aee855b8dc1eb277 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca00680d2a1f4210940ca453bd6bff68 in term 1.
I20260808 23:31:38.135319 4279 leader_election.cc:302] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca00680d2a1f4210940ca453bd6bff68, d5479a16d3194828aee855b8dc1eb277; no voters:
I20260808 23:31:38.135613 4666 raft_consensus.cc:2781] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:31:38.136023 4666 raft_consensus.cc:686] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68 [term 1 LEADER]: Becoming Leader. State: Replica: ca00680d2a1f4210940ca453bd6bff68, State: Running, Role: LEADER
I20260808 23:31:38.136435 4666 consensus_queue.cc:237] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5479a16d3194828aee855b8dc1eb277" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 33181 } } peers { permanent_uuid: "8a07026e5cbb4bf59402cf9c19918ab1" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38691 } } peers { permanent_uuid: "ca00680d2a1f4210940ca453bd6bff68" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 46027 } }
I20260808 23:31:38.140595 3985 catalog_manager.cc:5495] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca00680d2a1f4210940ca453bd6bff68 (127.9.44.66). New cstate: current_term: 1 leader_uuid: "ca00680d2a1f4210940ca453bd6bff68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5479a16d3194828aee855b8dc1eb277" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 33181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a07026e5cbb4bf59402cf9c19918ab1" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca00680d2a1f4210940ca453bd6bff68" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 46027 } health_report { overall_health: HEALTHY } } }
I20260808 23:31:38.174499 4656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:38.169151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38522 (local address 127.9.44.66:46027)
0808 23:31:38.169335 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:38.169342 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:38.169425 (+ 83us) server_negotiation.cc:407] Connection header received
0808 23:31:38.169636 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:38.169640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:38.169719 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:38.169841 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:31:38.170203 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.170875 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.172506 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.172939 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.173053 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:38.173541 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:31:38.173985 (+ 444us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:31:38.174108 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:38.174203 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 23:31:38.174275 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 23:31:38.182384 4558 raft_consensus.cc:1261] T 9eed985f4fda4d1985ad52bc437dcc60 P 8a07026e5cbb4bf59402cf9c19918ab1 [term 1 FOLLOWER]: Refusing update from remote peer ca00680d2a1f4210940ca453bd6bff68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:31:38.182389 4182 raft_consensus.cc:1261] T 9eed985f4fda4d1985ad52bc437dcc60 P d5479a16d3194828aee855b8dc1eb277 [term 1 FOLLOWER]: Refusing update from remote peer ca00680d2a1f4210940ca453bd6bff68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:31:38.183364 4666 consensus_queue.cc:1035] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5479a16d3194828aee855b8dc1eb277" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 33181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:31:38.183617 4671 consensus_queue.cc:1035] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a07026e5cbb4bf59402cf9c19918ab1" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 38691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:31:38.216260 3985 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:31:38.216539 3985 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:31:38.222064 3985 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8874b3ff80bc417b83b105fa12e5fad8: Sending DeleteTablet for 3 replicas of tablet 9eed985f4fda4d1985ad52bc437dcc60
I20260808 23:31:38.223403 4530 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eed985f4fda4d1985ad52bc437dcc60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:31:38 UTC) from {username='slave'} at 127.0.0.1:49212
I20260808 23:31:38.224098 4337 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eed985f4fda4d1985ad52bc437dcc60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:31:38 UTC) from {username='slave'} at 127.0.0.1:38508
I20260808 23:31:38.224932 4153 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eed985f4fda4d1985ad52bc437dcc60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:31:38 UTC) from {username='slave'} at 127.0.0.1:53576
I20260808 23:31:38.225590 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 4072
I20260808 23:31:38.230983 4689 tablet_replica.cc:330] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68: stopping tablet replica
I20260808 23:31:38.231426 4689 raft_consensus.cc:2229] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:31:38.232124 4689 raft_consensus.cc:2258] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:31:38.234803 4689 ts_tablet_manager.cc:1905] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:31:38.242420 4689 ts_tablet_manager.cc:1918] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:31:38.242651 4689 log.cc:1196] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/9eed985f4fda4d1985ad52bc437dcc60
I20260808 23:31:38.243218 4689 ts_tablet_manager.cc:1939] T 9eed985f4fda4d1985ad52bc437dcc60 P ca00680d2a1f4210940ca453bd6bff68: Deleting consensus metadata
I20260808 23:31:38.244975 3968 catalog_manager.cc:4841] TS ca00680d2a1f4210940ca453bd6bff68 (127.9.44.66:46027): tablet 9eed985f4fda4d1985ad52bc437dcc60 (table test-table [id=bc8fe013ce66498d915d65d089895875]) successfully deleted
I20260808 23:31:38.250296 4650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:38.227033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53590 (local address 127.9.44.65:33181)
0808 23:31:38.227203 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:38.227209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:38.232693 (+ 5484us) server_negotiation.cc:407] Connection header received
0808 23:31:38.238221 (+ 5528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:38.238227 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:38.238328 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:38.238411 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:38.239106 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.240459 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.242448 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.243138 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.243230 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:38.248134 (+ 4904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:38.248146 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:38.248164 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:38.248925 (+ 761us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:38.248987 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:38.249013 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:38.249128 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:38.249671 (+ 543us) server_negotiation.cc:299] Negotiation successful
0808 23:31:38.249786 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 23:31:38.250718 4687 tablet_replica.cc:330] T 9eed985f4fda4d1985ad52bc437dcc60 P 8a07026e5cbb4bf59402cf9c19918ab1: stopping tablet replica
I20260808 23:31:38.250782 4690 tablet_replica.cc:330] T 9eed985f4fda4d1985ad52bc437dcc60 P d5479a16d3194828aee855b8dc1eb277: stopping tablet replica
I20260808 23:31:38.251214 4243 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:38.251224 4687 raft_consensus.cc:2229] T 9eed985f4fda4d1985ad52bc437dcc60 P 8a07026e5cbb4bf59402cf9c19918ab1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:31:38.251835 4687 raft_consensus.cc:2258] T 9eed985f4fda4d1985ad52bc437dcc60 P 8a07026e5cbb4bf59402cf9c19918ab1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:31:38.255168 4687 ts_tablet_manager.cc:1905] T 9eed985f4fda4d1985ad52bc437dcc60 P 8a07026e5cbb4bf59402cf9c19918ab1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:31:38.262012 4687 ts_tablet_manager.cc:1918] T 9eed985f4fda4d1985ad52bc437dcc60 P 8a07026e5cbb4bf59402cf9c19918ab1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:31:38.262190 4687 log.cc:1196] T 9eed985f4fda4d1985ad52bc437dcc60 P 8a07026e5cbb4bf59402cf9c19918ab1: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/9eed985f4fda4d1985ad52bc437dcc60
I20260808 23:31:38.262709 4687 ts_tablet_manager.cc:1939] T 9eed985f4fda4d1985ad52bc437dcc60 P 8a07026e5cbb4bf59402cf9c19918ab1: Deleting consensus metadata
I20260808 23:31:38.264528 3968 catalog_manager.cc:4841] TS 8a07026e5cbb4bf59402cf9c19918ab1 (127.9.44.67:38691): tablet 9eed985f4fda4d1985ad52bc437dcc60 (table test-table [id=bc8fe013ce66498d915d65d089895875]) successfully deleted
I20260808 23:31:38.252461 4690 raft_consensus.cc:2229] T 9eed985f4fda4d1985ad52bc437dcc60 P d5479a16d3194828aee855b8dc1eb277 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:31:38.342823 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 4072
I20260808 23:31:38.342909 4690 raft_consensus.cc:2258] T 9eed985f4fda4d1985ad52bc437dcc60 P d5479a16d3194828aee855b8dc1eb277 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:31:38.366883 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 4258
W20260808 23:31:38.366966 3969 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.44.65:33181 (error 108)
I20260808 23:31:38.369217 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:38.368766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:33181 (local address 127.0.0.1:53604)
0808 23:31:38.368950 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:38.369074 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260808 23:31:38.370474 3969 catalog_manager.cc:4584] TS d5479a16d3194828aee855b8dc1eb277 (127.9.44.65:33181): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9eed985f4fda4d1985ad52bc437dcc60: Network error: Client connection negotiation failed: client connection to 127.9.44.65:33181: connect: Connection refused (error 111)
I20260808 23:31:38.375190 4656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:38.367763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38528 (local address 127.9.44.66:46027)
0808 23:31:38.367946 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:38.367952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:38.367974 (+ 22us) server_negotiation.cc:407] Connection header received
0808 23:31:38.368152 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:38.368158 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:38.368280 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:38.368384 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:38.369020 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.370216 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.371704 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.372380 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.372514 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:38.373597 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:38.373610 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:38.373628 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:38.374420 (+ 792us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:38.374481 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:38.374505 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:38.374605 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:38.374719 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 23:31:38.374826 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 23:31:38.376098 4425 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:38.424217 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:38.423747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:33181 (local address 127.0.0.1:53612)
0808 23:31:38.423943 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:38.424032 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 23:31:38.425278 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:38.424960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:33181 (local address 127.0.0.1:53624)
0808 23:31:38.425076 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:38.425146 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 23:31:38.484840 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 4258
I20260808 23:31:38.485666 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:38.485149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:33181 (local address 127.0.0.1:53630)
0808 23:31:38.485433 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:38.485513 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
I20260808 23:31:38.486642 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:38.486303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:33181 (local address 127.0.0.1:53632)
0808 23:31:38.486455 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:38.486533 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 23:31:38.512130 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 4439
I20260808 23:31:38.518400 4654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:38.512903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49220 (local address 127.9.44.67:38691)
0808 23:31:38.513059 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:38.513063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:38.513081 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:31:38.513245 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:38.513249 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:38.513336 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:38.513415 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:38.513733 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.514480 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.515689 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:38.516082 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:38.516176 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:38.517168 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:38.517177 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:38.517193 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:38.517754 (+ 561us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:38.517802 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:38.517821 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:38.517906 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:38.518018 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 23:31:38.518106 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 23:31:38.519021 4625 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:38.566917 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:38.566527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:33181 (local address 127.0.0.1:53636)
0808 23:31:38.566698 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:38.566782 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 23:31:38.567915 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:38.567512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:33181 (local address 127.0.0.1:53638)
0808 23:31:38.567732 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:38.567809 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:33181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 23:31:38.605422 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 4439
I20260808 23:31:38.632652 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 3947
I20260808 23:31:38.633749 4033 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:38.739894 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 3947
2026-08-08T23:31:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2722 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:31:38 dist-test-slave-6bbx krb5kdc[4701](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:31:38 dist-test-slave-6bbx krb5kdc[4701](info): set up 2 sockets
Aug 08 23:31:38 dist-test-slave-6bbx krb5kdc[4701](info): commencing operation
krb5kdc: starting...
W20260808 23:31:39.959426 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.004s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:31:40 dist-test-slave-6bbx krb5kdc[4701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:31:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:31:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:31:40.208774 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:42991
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:40989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:42991
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:31:40.530388 4720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:40.530665 4720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:40.530797 4720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:40.537603 4720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:31:40.537704 4720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:40.537781 4720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:40.537854 4720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:31:40.537914 4720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:31:40.547124 4720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:40989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:42991
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:42991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:40.548334 4720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:40.549580 4720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:40.560351 4726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:40.560385 4731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:40.560422 4729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:40.561424 4720 server_base.cc:1034] running on GCE node
I20260808 23:31:40.562076 4720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:40.563277 4720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:40.564465 4720 hybrid_clock.cc:648] HybridClock initialized: now 1786231900564429 us; error 24 us; skew 500 ppm
I20260808 23:31:40.564746 4720 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:31:40 dist-test-slave-6bbx krb5kdc[4701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231900, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:31:40.567533 4720 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:31:40.568878 4720 webserver.cc:460] Webserver started at http://127.9.44.126:35633/ using document root <none> and password file <none>
I20260808 23:31:40.569478 4720 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:40.569576 4720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:40.569818 4720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:40.572580 4720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "4ff9a46b29e24b8687902547ea8ba661"
format_stamp: "Formatted at 2026-08-08 23:31:40 on dist-test-slave-6bbx"
server_key: "ff86268ffa318238a9b1c3713848432e"
server_key_iv: "a2e70d8533ca749b2918b56d2a8dc984"
server_key_version: "encryptionkey@0"
I20260808 23:31:40.573297 4720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "4ff9a46b29e24b8687902547ea8ba661"
format_stamp: "Formatted at 2026-08-08 23:31:40 on dist-test-slave-6bbx"
server_key: "ff86268ffa318238a9b1c3713848432e"
server_key_iv: "a2e70d8533ca749b2918b56d2a8dc984"
server_key_version: "encryptionkey@0"
I20260808 23:31:40.577962 4720 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260808 23:31:40.581215 4738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:40.582388 4720 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 23:31:40.582542 4720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "4ff9a46b29e24b8687902547ea8ba661"
format_stamp: "Formatted at 2026-08-08 23:31:40 on dist-test-slave-6bbx"
server_key: "ff86268ffa318238a9b1c3713848432e"
server_key_iv: "a2e70d8533ca749b2918b56d2a8dc984"
server_key_version: "encryptionkey@0"
I20260808 23:31:40.582708 4720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:40.596686 4720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:40.600687 4720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:40.600927 4720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:40.616243 4720 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:42991
I20260808 23:31:40.616250 4808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:42991 every 8 connection(s)
I20260808 23:31:40.617421 4720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:31:40.620821 4809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:40.621114 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 4720
I20260808 23:31:40.621272 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:31:40.621657 9393 external_mini_cluster.cc:1428] Setting key d5ac0ca5d01ba812839be95b12626904
I20260808 23:31:40.633571 4809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ff9a46b29e24b8687902547ea8ba661: Bootstrap starting.
Aug 08 23:31:40 dist-test-slave-6bbx krb5kdc[4701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:31:40.637233 4809 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ff9a46b29e24b8687902547ea8ba661: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:40.638264 4809 log.cc:824] T 00000000000000000000000000000000 P 4ff9a46b29e24b8687902547ea8ba661: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:40.640996 4809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ff9a46b29e24b8687902547ea8ba661: No bootstrap required, opened a new log
I20260808 23:31:40.644321 4812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:40.623303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46698 (local address 127.9.44.126:42991)
0808 23:31:40.624124 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:40.624136 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:40.624172 (+ 36us) server_negotiation.cc:407] Connection header received
0808 23:31:40.626394 (+ 2222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:40.626415 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:40.626758 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:40.627060 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:40.628806 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:40.630059 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:40.631907 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:40.632526 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:40.632648 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:40.637433 (+ 4785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:40.637441 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:40.637454 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:40.637491 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:40.640383 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:40.641176 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:40.641185 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:40.641192 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:40.641363 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:40.641787 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:40.641791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:40.641792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:40.642301 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:40.642518 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:40.642968 (+ 450us) server_negotiation.cc:299] Negotiation successful
0808 23:31:40.643299 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":410,"threads_started":1}
I20260808 23:31:40.648720 4809 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ff9a46b29e24b8687902547ea8ba661 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ff9a46b29e24b8687902547ea8ba661" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 42991 } }
I20260808 23:31:40.649044 4809 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ff9a46b29e24b8687902547ea8ba661 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:40.649140 4809 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ff9a46b29e24b8687902547ea8ba661 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ff9a46b29e24b8687902547ea8ba661, State: Initialized, Role: FOLLOWER
I20260808 23:31:40.649626 4809 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ff9a46b29e24b8687902547ea8ba661 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ff9a46b29e24b8687902547ea8ba661" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 42991 } }
I20260808 23:31:40.649840 4809 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ff9a46b29e24b8687902547ea8ba661 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:31:40.649977 4809 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ff9a46b29e24b8687902547ea8ba661 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:31:40.650121 4809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ff9a46b29e24b8687902547ea8ba661 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:40.652170 4809 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ff9a46b29e24b8687902547ea8ba661 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ff9a46b29e24b8687902547ea8ba661" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 42991 } }
I20260808 23:31:40.652671 4809 leader_election.cc:302] T 00000000000000000000000000000000 P 4ff9a46b29e24b8687902547ea8ba661 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ff9a46b29e24b8687902547ea8ba661; no voters:
I20260808 23:31:40.653226 4809 leader_election.cc:288] T 00000000000000000000000000000000 P 4ff9a46b29e24b8687902547ea8ba661 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:31:40.653342 4816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ff9a46b29e24b8687902547ea8ba661 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:31:40.653986 4816 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ff9a46b29e24b8687902547ea8ba661 [term 1 LEADER]: Becoming Leader. State: Replica: 4ff9a46b29e24b8687902547ea8ba661, State: Running, Role: LEADER
I20260808 23:31:40.654487 4816 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ff9a46b29e24b8687902547ea8ba661 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ff9a46b29e24b8687902547ea8ba661" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 42991 } }
I20260808 23:31:40.655055 4809 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ff9a46b29e24b8687902547ea8ba661 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:31:40.657048 4817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ff9a46b29e24b8687902547ea8ba661 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ff9a46b29e24b8687902547ea8ba661" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ff9a46b29e24b8687902547ea8ba661" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 42991 } } }
I20260808 23:31:40.657058 4818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ff9a46b29e24b8687902547ea8ba661 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ff9a46b29e24b8687902547ea8ba661. Latest consensus state: current_term: 1 leader_uuid: "4ff9a46b29e24b8687902547ea8ba661" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ff9a46b29e24b8687902547ea8ba661" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 42991 } } }
I20260808 23:31:40.657478 4817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ff9a46b29e24b8687902547ea8ba661 [sys.catalog]: This master's current role is: LEADER
I20260808 23:31:40.657502 4818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ff9a46b29e24b8687902547ea8ba661 [sys.catalog]: This master's current role is: LEADER
I20260808 23:31:40.663789 4824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:31:40.669085 4824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:31:40.676358 4824 catalog_manager.cc:1347] Generated new cluster ID: b022acbcc76140bdaf927a1fdf3cf7c6
I20260808 23:31:40.676455 4824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:31:40.699136 4824 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:31:40.700351 4824 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:31:40.709513 4824 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ff9a46b29e24b8687902547ea8ba661: Generated new TSK 0
I20260808 23:31:40.710228 4824 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.44.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
I20260808 23:31:40.780633 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:0
--local_ip_for_outbound_sockets=127.9.44.65
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:42991
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:40989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:31:41.103947 4844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:41.104180 4844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:41.104279 4844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:41.111057 4844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:41.111150 4844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:41.111259 4844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:31:41.120302 4844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:40989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:42991
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:41.121338 4844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:41.122603 4844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:41.134662 4850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:41.134886 4849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:41.134935 4852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:41.135529 4844 server_base.cc:1034] running on GCE node
I20260808 23:31:41.136122 4844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:41.136962 4844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:41.138155 4844 hybrid_clock.cc:648] HybridClock initialized: now 1786231901138110 us; error 34 us; skew 500 ppm
I20260808 23:31:41.138446 4844 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:31:41 dist-test-slave-6bbx krb5kdc[4701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231901, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:31:41.141355 4844 init.cc:377] Logged in from keytab as kudu/127.9.44.65@KRBTEST.COM (short username kudu)
I20260808 23:31:41.142715 4844 webserver.cc:460] Webserver started at http://127.9.44.65:44963/ using document root <none> and password file <none>
I20260808 23:31:41.143352 4844 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:41.143476 4844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:41.143803 4844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:41.146510 4844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-0/data/instance:
uuid: "3efef993c88448b98b55cff6318efc26"
format_stamp: "Formatted at 2026-08-08 23:31:41 on dist-test-slave-6bbx"
server_key: "2922f84913d6251efb2913a6cadfe3e6"
server_key_iv: "699fabda20a06cb81a6f164125ea2f60"
server_key_version: "encryptionkey@0"
I20260808 23:31:41.147241 4844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance:
uuid: "3efef993c88448b98b55cff6318efc26"
format_stamp: "Formatted at 2026-08-08 23:31:41 on dist-test-slave-6bbx"
server_key: "2922f84913d6251efb2913a6cadfe3e6"
server_key_iv: "699fabda20a06cb81a6f164125ea2f60"
server_key_version: "encryptionkey@0"
I20260808 23:31:41.151974 4844 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260808 23:31:41.155203 4861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:41.156451 4844 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:31:41.156594 4844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "3efef993c88448b98b55cff6318efc26"
format_stamp: "Formatted at 2026-08-08 23:31:41 on dist-test-slave-6bbx"
server_key: "2922f84913d6251efb2913a6cadfe3e6"
server_key_iv: "699fabda20a06cb81a6f164125ea2f60"
server_key_version: "encryptionkey@0"
I20260808 23:31:41.156771 4844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:41.189007 4844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:41.192586 4844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:41.192840 4844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:41.193723 4844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:31:41.194975 4844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:31:41.195052 4844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:41.195173 4844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:31:41.195227 4844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:41.224534 4844 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:42231
I20260808 23:31:41.224593 5019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:42231 every 8 connection(s)
I20260808 23:31:41.225714 4844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:31:41.233686 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 4844
I20260808 23:31:41.233893 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance
I20260808 23:31:41.234436 9393 external_mini_cluster.cc:1428] Setting key 0308d26339fc0f34d103398ce0f5c9cc
Aug 08 23:31:41 dist-test-slave-6bbx krb5kdc[4701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231901, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:31:41.245160 5022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:41.228026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:39449 (local address 127.9.44.126:42991)
0808 23:31:41.228413 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:41.228417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:41.229369 (+ 952us) server_negotiation.cc:407] Connection header received
0808 23:31:41.231104 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:41.231108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:41.231173 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:41.231267 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:41.232694 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:41.233421 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:41.235464 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:41.235928 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:41.236021 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:41.239516 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:41.239522 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:41.239526 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:41.239563 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:41.242398 (+ 2835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:41.243041 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:41.243047 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:41.243050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:41.243189 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:41.243564 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:41.243568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:41.243571 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:41.243899 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:41.244030 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:41.244643 (+ 613us) server_negotiation.cc:299] Negotiation successful
0808 23:31:41.244843 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":186,"threads_started":1}
I20260808 23:31:41.245851 5023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:41.228557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:42991 (local address 127.9.44.65:39449)
0808 23:31:41.229214 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:41.229259 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:41.230866 (+ 1607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:41.231455 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:41.231462 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:41.232028 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:31:41.232559 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:41.232577 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:41.233586 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:41.233591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:41.235312 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:41.235325 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:41.236151 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:41.236160 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:41.236455 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:41.237078 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:31:41.237104 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:31:41.239305 (+ 2201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:31:41.242522 (+ 3217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:31:41.242528 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:31:41.242538 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:31:41.242934 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:31:41.243291 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:31:41.243294 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:31:41.243296 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:31:41.243466 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:31:41.244000 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:31:41.244006 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:31:41.244287 (+ 281us) client_negotiation.cc:769] Sending connection context
0808 23:31:41.244507 (+ 220us) client_negotiation.cc:240] Negotiation successful
0808 23:31:41.244850 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":265,"threads_started":1}
I20260808 23:31:41.247249 5020 heartbeater.cc:347] Connected to a master server at 127.9.44.126:42991
I20260808 23:31:41.247529 5020 heartbeater.cc:464] Registering TS with master...
I20260808 23:31:41.248224 5020 heartbeater.cc:511] Master 127.9.44.126:42991 requested a full tablet report, sending...
I20260808 23:31:41.250327 4761 ts_manager.cc:194] Registered new tserver with Master: 3efef993c88448b98b55cff6318efc26 (127.9.44.65:42231)
I20260808 23:31:41.251745 4761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.65@KRBTEST.COM'} at 127.9.44.65:39449
WARNING: no policy specified for kudu/127.9.44.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
I20260808 23:31:41.294160 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:0
--local_ip_for_outbound_sockets=127.9.44.66
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:42991
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:40989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:31:41.616245 5027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:41.616496 5027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:41.616634 5027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:41.623445 5027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:41.623551 5027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:41.623737 5027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:31:41.632877 5027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:40989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:42991
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:41.633951 5027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:41.635185 5027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:41.647426 5041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:41.647361 5037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:41.647461 5036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:41.648308 5027 server_base.cc:1034] running on GCE node
I20260808 23:31:41.648900 5027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:41.649755 5027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:41.650949 5027 hybrid_clock.cc:648] HybridClock initialized: now 1786231901650895 us; error 46 us; skew 500 ppm
I20260808 23:31:41.651239 5027 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:31:41 dist-test-slave-6bbx krb5kdc[4701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231901, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:31:41.658250 5027 init.cc:377] Logged in from keytab as kudu/127.9.44.66@KRBTEST.COM (short username kudu)
I20260808 23:31:41.659622 5027 webserver.cc:460] Webserver started at http://127.9.44.66:33599/ using document root <none> and password file <none>
I20260808 23:31:41.660351 5027 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:41.660470 5027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:41.660768 5027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:41.663519 5027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-1/data/instance:
uuid: "c7dcfeb49e8440f98e7957c68f3c376b"
format_stamp: "Formatted at 2026-08-08 23:31:41 on dist-test-slave-6bbx"
server_key: "60e59f0a92360dc52ac3971b443f8d6d"
server_key_iv: "b708911c9d8838cae291561727de79b4"
server_key_version: "encryptionkey@0"
I20260808 23:31:41.664345 5027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance:
uuid: "c7dcfeb49e8440f98e7957c68f3c376b"
format_stamp: "Formatted at 2026-08-08 23:31:41 on dist-test-slave-6bbx"
server_key: "60e59f0a92360dc52ac3971b443f8d6d"
server_key_iv: "b708911c9d8838cae291561727de79b4"
server_key_version: "encryptionkey@0"
I20260808 23:31:41.669055 5027 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260808 23:31:41.672168 5050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:41.673332 5027 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260808 23:31:41.673473 5027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "c7dcfeb49e8440f98e7957c68f3c376b"
format_stamp: "Formatted at 2026-08-08 23:31:41 on dist-test-slave-6bbx"
server_key: "60e59f0a92360dc52ac3971b443f8d6d"
server_key_iv: "b708911c9d8838cae291561727de79b4"
server_key_version: "encryptionkey@0"
I20260808 23:31:41.673651 5027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:41.686996 5027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:41.690768 5027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:41.691010 5027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:41.691886 5027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:31:41.693063 5027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:31:41.693149 5027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:41.693264 5027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:31:41.693321 5027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:41.722854 5027 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:40357
I20260808 23:31:41.722895 5210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:40357 every 8 connection(s)
I20260808 23:31:41.724073 5027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:31:41.727820 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 5027
I20260808 23:31:41.728010 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance
I20260808 23:31:41.728560 9393 external_mini_cluster.cc:1428] Setting key 4acfb520b81c27ef00e9bd316e15a747
Aug 08 23:31:41 dist-test-slave-6bbx krb5kdc[4701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231901, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:31:41.744766 5022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:41.726424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:33377 (local address 127.9.44.126:42991)
0808 23:31:41.726595 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:41.726600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:41.727846 (+ 1246us) server_negotiation.cc:407] Connection header received
0808 23:31:41.729982 (+ 2136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:41.729988 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:41.730062 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:41.730177 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:41.732110 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:41.732845 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:41.734862 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:41.735265 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:41.735350 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:41.739217 (+ 3867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:41.739223 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:41.739226 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:41.739259 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:41.741722 (+ 2463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:41.742505 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:41.742508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:41.742511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:41.742634 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:41.743053 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:41.743056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:41.743058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:41.743312 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:41.743457 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:41.744181 (+ 724us) server_negotiation.cc:299] Negotiation successful
0808 23:31:41.744428 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 23:31:41.745673 5214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:41.726872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:42991 (local address 127.9.44.66:33377)
0808 23:31:41.727650 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:41.727728 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:41.729738 (+ 2010us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:41.730409 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:41.730417 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:41.731131 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:31:41.731914 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:41.731939 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:41.733003 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:41.733008 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:41.734740 (+ 1732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:41.734754 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:41.735473 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:41.735479 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:41.735764 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:41.736415 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:31:41.736441 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:31:41.738996 (+ 2555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:31:41.741864 (+ 2868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:31:41.741873 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:31:41.741886 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:31:41.742403 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:31:41.742732 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:31:41.742736 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:31:41.742738 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:31:41.742964 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:31:41.743452 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:31:41.743461 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:31:41.743801 (+ 340us) client_negotiation.cc:769] Sending connection context
0808 23:31:41.744040 (+ 239us) client_negotiation.cc:240] Negotiation successful
0808 23:31:41.744531 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":316,"threads_started":1}
I20260808 23:31:41.746999 5211 heartbeater.cc:347] Connected to a master server at 127.9.44.126:42991
I20260808 23:31:41.747359 5211 heartbeater.cc:464] Registering TS with master...
I20260808 23:31:41.748200 5211 heartbeater.cc:511] Master 127.9.44.126:42991 requested a full tablet report, sending...
I20260808 23:31:41.749794 4761 ts_manager.cc:194] Registered new tserver with Master: c7dcfeb49e8440f98e7957c68f3c376b (127.9.44.66:40357)
I20260808 23:31:41.750875 4761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.66@KRBTEST.COM'} at 127.9.44.66:33377
WARNING: no policy specified for kudu/127.9.44.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
I20260808 23:31:41.792079 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:0
--local_ip_for_outbound_sockets=127.9.44.67
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:42991
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:40989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:31:42.116544 5220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:42.116791 5220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:42.116889 5220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:42.123736 5220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:42.123821 5220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:42.123932 5220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:31:42.133059 5220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:40989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:42991
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:42.134187 5220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:42.135382 5220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:42.147442 5225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:42.147565 5226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:42.147601 5229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:42.148380 5220 server_base.cc:1034] running on GCE node
I20260808 23:31:42.148891 5220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:42.149693 5220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:42.150887 5220 hybrid_clock.cc:648] HybridClock initialized: now 1786231902150837 us; error 43 us; skew 500 ppm
I20260808 23:31:42.151162 5220 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:31:42 dist-test-slave-6bbx krb5kdc[4701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231902, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:31:42.154070 5220 init.cc:377] Logged in from keytab as kudu/127.9.44.67@KRBTEST.COM (short username kudu)
I20260808 23:31:42.155403 5220 webserver.cc:460] Webserver started at http://127.9.44.67:36343/ using document root <none> and password file <none>
I20260808 23:31:42.156081 5220 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:42.156196 5220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:42.156489 5220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:42.159119 5220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-2/data/instance:
uuid: "0af5d786c11048bd901dcbe9aeea4498"
format_stamp: "Formatted at 2026-08-08 23:31:42 on dist-test-slave-6bbx"
server_key: "fc2ce06b6729759e1b737a44c5b33fcb"
server_key_iv: "711f2883fa11578b69f0dd34bd530195"
server_key_version: "encryptionkey@0"
I20260808 23:31:42.160024 5220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance:
uuid: "0af5d786c11048bd901dcbe9aeea4498"
format_stamp: "Formatted at 2026-08-08 23:31:42 on dist-test-slave-6bbx"
server_key: "fc2ce06b6729759e1b737a44c5b33fcb"
server_key_iv: "711f2883fa11578b69f0dd34bd530195"
server_key_version: "encryptionkey@0"
I20260808 23:31:42.164799 5220 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 23:31:42.168053 5241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:42.169206 5220 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260808 23:31:42.169348 5220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "0af5d786c11048bd901dcbe9aeea4498"
format_stamp: "Formatted at 2026-08-08 23:31:42 on dist-test-slave-6bbx"
server_key: "fc2ce06b6729759e1b737a44c5b33fcb"
server_key_iv: "711f2883fa11578b69f0dd34bd530195"
server_key_version: "encryptionkey@0"
I20260808 23:31:42.169508 5220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:42.192050 5220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:42.195477 5220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:42.195741 5220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:42.196585 5220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:31:42.197777 5220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:31:42.197847 5220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:42.197964 5220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:31:42.198014 5220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:42.228091 5220 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:35205
I20260808 23:31:42.228158 5405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:35205 every 8 connection(s)
I20260808 23:31:42.229267 5220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:31:42.234678 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 5220
I20260808 23:31:42.234872 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance
I20260808 23:31:42.235286 9393 external_mini_cluster.cc:1428] Setting key d606ca414d035fb43159506eef9915e1
Aug 08 23:31:42 dist-test-slave-6bbx krb5kdc[4701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231902, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:31:42.247615 5022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:42.231569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:45023 (local address 127.9.44.126:42991)
0808 23:31:42.231762 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:42.231767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:42.232797 (+ 1030us) server_negotiation.cc:407] Connection header received
0808 23:31:42.234607 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:42.234613 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:42.234698 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:42.234788 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:42.236407 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.237394 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.239046 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.239461 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.239558 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:42.242644 (+ 3086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:42.242648 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:42.242652 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:42.242684 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:42.245190 (+ 2506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:42.245797 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:42.245800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:42.245802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:42.245926 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:42.246248 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:42.246251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:42.246253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:42.246495 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:42.246604 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:42.247231 (+ 627us) server_negotiation.cc:299] Negotiation successful
0808 23:31:42.247382 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 23:31:42.248452 5409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:42.231978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:42991 (local address 127.9.44.67:45023)
0808 23:31:42.232650 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:42.232690 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:42.234343 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:42.235046 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:42.235054 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:42.235574 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:31:42.236236 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:42.236254 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.237549 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.237555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:42.238917 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:42.238930 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.239629 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.239634 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:42.239876 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:42.240430 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:31:42.240451 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:31:42.242447 (+ 1996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:31:42.245300 (+ 2853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:31:42.245307 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:31:42.245317 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:31:42.245707 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:31:42.245999 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:31:42.246002 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:31:42.246004 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:31:42.246168 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:31:42.246636 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:31:42.246643 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:31:42.246926 (+ 283us) client_negotiation.cc:769] Sending connection context
0808 23:31:42.247151 (+ 225us) client_negotiation.cc:240] Negotiation successful
0808 23:31:42.247502 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":265,"threads_started":1}
I20260808 23:31:42.249738 5407 heartbeater.cc:347] Connected to a master server at 127.9.44.126:42991
I20260808 23:31:42.250063 5407 heartbeater.cc:464] Registering TS with master...
I20260808 23:31:42.250706 5407 heartbeater.cc:511] Master 127.9.44.126:42991 requested a full tablet report, sending...
I20260808 23:31:42.252002 4761 ts_manager.cc:194] Registered new tserver with Master: 0af5d786c11048bd901dcbe9aeea4498 (127.9.44.67:35205)
I20260808 23:31:42.252770 4761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.67@KRBTEST.COM'} at 127.9.44.67:45023
I20260808 23:31:42.254360 5020 heartbeater.cc:503] Master 127.9.44.126:42991 was elected leader, sending a full tablet report...
I20260808 23:31:42.261853 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:31:42.277866 5022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:42.265702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46708 (local address 127.9.44.126:42991)
0808 23:31:42.265842 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:42.265846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:42.266290 (+ 444us) server_negotiation.cc:407] Connection header received
0808 23:31:42.266440 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:42.266443 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:42.266496 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:42.266562 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:42.267879 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.268548 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.269770 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.270160 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.270243 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:42.272739 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:42.272744 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:42.272747 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:42.272775 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:42.275121 (+ 2346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:42.275914 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:42.275919 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:42.275921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:42.276039 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:42.276475 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:42.276479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:42.276481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:42.276807 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:42.277051 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:42.277411 (+ 360us) server_negotiation.cc:299] Negotiation successful
0808 23:31:42.277574 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 23:31:42.281031 4761 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46708:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:31:42.287719 4761 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:31:42.325498 5420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:42.298243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:42231 (local address 127.0.0.1:34550)
0808 23:31:42.313767 (+ 15524us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:42.313809 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:42.313971 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:42.318649 (+ 4678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:42.318653 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:42.318677 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:42.318980 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:42.318989 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.320540 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.320545 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:42.323045 (+ 2500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:42.323055 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.324512 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.324518 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:42.324707 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:42.324760 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 23:31:42.324892 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 23:31:42.325084 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15308,"mutex_wait_us":93,"thread_start_us":15163,"threads_started":1}
I20260808 23:31:42.325976 5424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:42.298645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:40357 (local address 127.0.0.1:58884)
0808 23:31:42.315715 (+ 17070us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:42.315733 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:42.315858 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:42.318034 (+ 2176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:42.318042 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:42.318076 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:42.318580 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:42.318591 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.320337 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.320342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:42.323014 (+ 2672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:42.323033 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.325094 (+ 2061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.325104 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:42.325311 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:42.325343 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 23:31:42.325460 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 23:31:42.325587 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16922,"mutex_wait_us":106,"thread_start_us":19474,"threads_started":1}
I20260808 23:31:42.326009 5425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:42.299184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58884 (local address 127.9.44.66:40357)
0808 23:31:42.315255 (+ 16071us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:42.315265 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:42.315989 (+ 724us) server_negotiation.cc:407] Connection header received
0808 23:31:42.316116 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:42.316125 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:42.316389 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:42.316581 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:42.318758 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.320194 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.323176 (+ 2982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.324662 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.324765 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:42.324926 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:42.325537 (+ 611us) server_negotiation.cc:299] Negotiation successful
0808 23:31:42.325703 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15901,"thread_start_us":15761,"threads_started":1}
I20260808 23:31:42.326333 5422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:42.298601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34550 (local address 127.9.44.65:42231)
0808 23:31:42.318109 (+ 19508us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:42.318118 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:42.318144 (+ 26us) server_negotiation.cc:407] Connection header received
0808 23:31:42.318232 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:42.318239 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:42.318449 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:42.318618 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:42.319115 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.320372 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.323191 (+ 2819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.324146 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.324229 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:42.324386 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:42.325890 (+ 1504us) server_negotiation.cc:299] Negotiation successful
0808 23:31:42.326051 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19333,"thread_start_us":19132,"threads_started":1}
I20260808 23:31:42.326380 5423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:42.298986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:35205 (local address 127.0.0.1:34458)
0808 23:31:42.318419 (+ 19433us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:42.318438 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:42.318572 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:42.319187 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:42.319192 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:42.319217 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:42.319568 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:42.319580 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.321439 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.321443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:42.323045 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:42.323058 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.325120 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.325125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:42.325552 (+ 427us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:42.325811 (+ 259us) client_negotiation.cc:769] Sending connection context
0808 23:31:42.325919 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 23:31:42.326040 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19293,"mutex_wait_us":314,"thread_start_us":16440,"threads_started":1}
I20260808 23:31:42.327164 5427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:42.299538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34458 (local address 127.9.44.67:35205)
0808 23:31:42.317429 (+ 17891us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:42.317437 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:42.318552 (+ 1115us) server_negotiation.cc:407] Connection header received
0808 23:31:42.318781 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:42.318789 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:42.319018 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:42.319206 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:42.319775 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.321270 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.323227 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.324619 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.324746 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:42.324980 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:42.326676 (+ 1696us) server_negotiation.cc:299] Negotiation successful
0808 23:31:42.326833 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17716,"thread_start_us":17891,"threads_started":1}
I20260808 23:31:42.329733 5121 tablet_service.cc:1467] Processing CreateTablet for tablet 09dfcb4a9086489a8248f0ba38e36619 (DEFAULT_TABLE table=test-table [id=da7d3b7da0144a6ca4daa7259141c23f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:31:42.329725 4928 tablet_service.cc:1467] Processing CreateTablet for tablet 09dfcb4a9086489a8248f0ba38e36619 (DEFAULT_TABLE table=test-table [id=da7d3b7da0144a6ca4daa7259141c23f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:31:42.331418 5121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09dfcb4a9086489a8248f0ba38e36619. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:42.331485 4928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09dfcb4a9086489a8248f0ba38e36619. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:42.331749 5309 tablet_service.cc:1467] Processing CreateTablet for tablet 09dfcb4a9086489a8248f0ba38e36619 (DEFAULT_TABLE table=test-table [id=da7d3b7da0144a6ca4daa7259141c23f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:31:42.333022 5309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09dfcb4a9086489a8248f0ba38e36619. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:42.344272 5429 tablet_bootstrap.cc:492] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26: Bootstrap starting.
I20260808 23:31:42.346115 5430 tablet_bootstrap.cc:492] T 09dfcb4a9086489a8248f0ba38e36619 P c7dcfeb49e8440f98e7957c68f3c376b: Bootstrap starting.
I20260808 23:31:42.347471 5432 tablet_bootstrap.cc:492] T 09dfcb4a9086489a8248f0ba38e36619 P 0af5d786c11048bd901dcbe9aeea4498: Bootstrap starting.
I20260808 23:31:42.347900 5429 tablet_bootstrap.cc:654] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:42.349192 5429 log.cc:824] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:42.349627 5430 tablet_bootstrap.cc:654] T 09dfcb4a9086489a8248f0ba38e36619 P c7dcfeb49e8440f98e7957c68f3c376b: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:42.350217 5432 tablet_bootstrap.cc:654] T 09dfcb4a9086489a8248f0ba38e36619 P 0af5d786c11048bd901dcbe9aeea4498: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:42.350708 5430 log.cc:824] T 09dfcb4a9086489a8248f0ba38e36619 P c7dcfeb49e8440f98e7957c68f3c376b: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:42.351219 5432 log.cc:824] T 09dfcb4a9086489a8248f0ba38e36619 P 0af5d786c11048bd901dcbe9aeea4498: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:42.352813 5429 tablet_bootstrap.cc:492] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26: No bootstrap required, opened a new log
I20260808 23:31:42.353108 5429 ts_tablet_manager.cc:1397] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260808 23:31:42.353744 5432 tablet_bootstrap.cc:492] T 09dfcb4a9086489a8248f0ba38e36619 P 0af5d786c11048bd901dcbe9aeea4498: No bootstrap required, opened a new log
I20260808 23:31:42.353973 5432 ts_tablet_manager.cc:1397] T 09dfcb4a9086489a8248f0ba38e36619 P 0af5d786c11048bd901dcbe9aeea4498: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s
I20260808 23:31:42.360888 5430 tablet_bootstrap.cc:492] T 09dfcb4a9086489a8248f0ba38e36619 P c7dcfeb49e8440f98e7957c68f3c376b: No bootstrap required, opened a new log
I20260808 23:31:42.361191 5430 ts_tablet_manager.cc:1397] T 09dfcb4a9086489a8248f0ba38e36619 P c7dcfeb49e8440f98e7957c68f3c376b: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.012s
I20260808 23:31:42.361377 5432 raft_consensus.cc:348] T 09dfcb4a9086489a8248f0ba38e36619 P 0af5d786c11048bd901dcbe9aeea4498 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3efef993c88448b98b55cff6318efc26" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42231 } } peers { permanent_uuid: "c7dcfeb49e8440f98e7957c68f3c376b" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 40357 } } peers { permanent_uuid: "0af5d786c11048bd901dcbe9aeea4498" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 35205 } }
I20260808 23:31:42.361693 5432 raft_consensus.cc:374] T 09dfcb4a9086489a8248f0ba38e36619 P 0af5d786c11048bd901dcbe9aeea4498 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:42.361788 5432 raft_consensus.cc:729] T 09dfcb4a9086489a8248f0ba38e36619 P 0af5d786c11048bd901dcbe9aeea4498 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0af5d786c11048bd901dcbe9aeea4498, State: Initialized, Role: FOLLOWER
I20260808 23:31:42.362401 5432 consensus_queue.cc:260] T 09dfcb4a9086489a8248f0ba38e36619 P 0af5d786c11048bd901dcbe9aeea4498 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3efef993c88448b98b55cff6318efc26" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42231 } } peers { permanent_uuid: "c7dcfeb49e8440f98e7957c68f3c376b" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 40357 } } peers { permanent_uuid: "0af5d786c11048bd901dcbe9aeea4498" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 35205 } }
I20260808 23:31:42.363648 5429 raft_consensus.cc:348] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3efef993c88448b98b55cff6318efc26" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42231 } } peers { permanent_uuid: "c7dcfeb49e8440f98e7957c68f3c376b" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 40357 } } peers { permanent_uuid: "0af5d786c11048bd901dcbe9aeea4498" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 35205 } }
I20260808 23:31:42.364105 5429 raft_consensus.cc:374] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:42.364202 5429 raft_consensus.cc:729] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3efef993c88448b98b55cff6318efc26, State: Initialized, Role: FOLLOWER
I20260808 23:31:42.364759 5429 consensus_queue.cc:260] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3efef993c88448b98b55cff6318efc26" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42231 } } peers { permanent_uuid: "c7dcfeb49e8440f98e7957c68f3c376b" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 40357 } } peers { permanent_uuid: "0af5d786c11048bd901dcbe9aeea4498" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 35205 } }
I20260808 23:31:42.366441 5429 ts_tablet_manager.cc:1428] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26: Time spent starting tablet: real 0.013s user 0.003s sys 0.010s
I20260808 23:31:42.367319 5407 heartbeater.cc:503] Master 127.9.44.126:42991 was elected leader, sending a full tablet report...
I20260808 23:31:42.368494 5432 ts_tablet_manager.cc:1428] T 09dfcb4a9086489a8248f0ba38e36619 P 0af5d786c11048bd901dcbe9aeea4498: Time spent starting tablet: real 0.014s user 0.011s sys 0.003s
I20260808 23:31:42.371263 5430 raft_consensus.cc:348] T 09dfcb4a9086489a8248f0ba38e36619 P c7dcfeb49e8440f98e7957c68f3c376b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3efef993c88448b98b55cff6318efc26" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42231 } } peers { permanent_uuid: "c7dcfeb49e8440f98e7957c68f3c376b" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 40357 } } peers { permanent_uuid: "0af5d786c11048bd901dcbe9aeea4498" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 35205 } }
I20260808 23:31:42.371696 5430 raft_consensus.cc:374] T 09dfcb4a9086489a8248f0ba38e36619 P c7dcfeb49e8440f98e7957c68f3c376b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:42.371804 5430 raft_consensus.cc:729] T 09dfcb4a9086489a8248f0ba38e36619 P c7dcfeb49e8440f98e7957c68f3c376b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7dcfeb49e8440f98e7957c68f3c376b, State: Initialized, Role: FOLLOWER
I20260808 23:31:42.372298 5430 consensus_queue.cc:260] T 09dfcb4a9086489a8248f0ba38e36619 P c7dcfeb49e8440f98e7957c68f3c376b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3efef993c88448b98b55cff6318efc26" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42231 } } peers { permanent_uuid: "c7dcfeb49e8440f98e7957c68f3c376b" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 40357 } } peers { permanent_uuid: "0af5d786c11048bd901dcbe9aeea4498" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 35205 } }
I20260808 23:31:42.373231 5211 heartbeater.cc:503] Master 127.9.44.126:42991 was elected leader, sending a full tablet report...
I20260808 23:31:42.373749 5430 ts_tablet_manager.cc:1428] T 09dfcb4a9086489a8248f0ba38e36619 P c7dcfeb49e8440f98e7957c68f3c376b: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260808 23:31:42.415839 5438 raft_consensus.cc:482] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:31:42.416115 5438 raft_consensus.cc:504] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3efef993c88448b98b55cff6318efc26" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42231 } } peers { permanent_uuid: "c7dcfeb49e8440f98e7957c68f3c376b" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 40357 } } peers { permanent_uuid: "0af5d786c11048bd901dcbe9aeea4498" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 35205 } }
I20260808 23:31:42.417685 5438 leader_election.cc:288] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7dcfeb49e8440f98e7957c68f3c376b (127.9.44.66:40357), 0af5d786c11048bd901dcbe9aeea4498 (127.9.44.67:35205)
I20260808 23:31:42.426247 5425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:42.418044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:42081 (local address 127.9.44.66:40357)
0808 23:31:42.418235 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:42.418242 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:42.419057 (+ 815us) server_negotiation.cc:407] Connection header received
0808 23:31:42.419239 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:42.419244 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:42.419320 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:42.419408 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:42.420201 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.421243 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.424042 (+ 2799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.425216 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.425349 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:42.425440 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:42.425873 (+ 433us) server_negotiation.cc:299] Negotiation successful
0808 23:31:42.425998 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 23:31:42.426247 5427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:42.418088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:40631 (local address 127.9.44.67:35205)
0808 23:31:42.418248 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:42.418253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:42.419050 (+ 797us) server_negotiation.cc:407] Connection header received
0808 23:31:42.419275 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:42.419280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:42.419382 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:42.419461 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:42.420135 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.421243 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.424040 (+ 2797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.425227 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.425356 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:42.425446 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:42.425873 (+ 427us) server_negotiation.cc:299] Negotiation successful
0808 23:31:42.425998 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 23:31:42.426473 5441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:42.418098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:40357 (local address 127.9.44.65:42081)
0808 23:31:42.418957 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:42.418973 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:42.419098 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:42.419437 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:42.419442 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:42.419471 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:42.420034 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:42.420044 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.421362 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.421368 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:42.423908 (+ 2540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:42.423918 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.425498 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.425505 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:42.425694 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:42.425740 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 23:31:42.425830 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 23:31:42.426016 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"mutex_wait_us":161,"thread_start_us":408,"threads_started":1}
I20260808 23:31:42.426496 5442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:42.418082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:35205 (local address 127.9.44.65:40631)
0808 23:31:42.418920 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:42.418950 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:42.419142 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:42.419474 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:42.419478 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:42.419495 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:42.419992 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:42.420011 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.421382 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.421386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:42.423887 (+ 2501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:42.423907 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.425465 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.425476 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:42.425687 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:42.425737 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 23:31:42.425828 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 23:31:42.426016 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"mutex_wait_us":123,"spinlock_wait_cycles":34560,"thread_start_us":452,"threads_started":1}
I20260808 23:31:42.427412 5149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09dfcb4a9086489a8248f0ba38e36619" candidate_uuid: "3efef993c88448b98b55cff6318efc26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7dcfeb49e8440f98e7957c68f3c376b" is_pre_election: true
I20260808 23:31:42.427619 5340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09dfcb4a9086489a8248f0ba38e36619" candidate_uuid: "3efef993c88448b98b55cff6318efc26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0af5d786c11048bd901dcbe9aeea4498" is_pre_election: true
I20260808 23:31:42.427881 5149 raft_consensus.cc:2454] T 09dfcb4a9086489a8248f0ba38e36619 P c7dcfeb49e8440f98e7957c68f3c376b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3efef993c88448b98b55cff6318efc26 in term 0.
I20260808 23:31:42.428113 5340 raft_consensus.cc:2454] T 09dfcb4a9086489a8248f0ba38e36619 P 0af5d786c11048bd901dcbe9aeea4498 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3efef993c88448b98b55cff6318efc26 in term 0.
I20260808 23:31:42.428580 4865 leader_election.cc:302] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3efef993c88448b98b55cff6318efc26, c7dcfeb49e8440f98e7957c68f3c376b; no voters:
I20260808 23:31:42.428912 5438 raft_consensus.cc:2781] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:31:42.429057 5438 raft_consensus.cc:482] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:31:42.429162 5438 raft_consensus.cc:3037] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:42.431214 5438 raft_consensus.cc:504] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3efef993c88448b98b55cff6318efc26" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42231 } } peers { permanent_uuid: "c7dcfeb49e8440f98e7957c68f3c376b" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 40357 } } peers { permanent_uuid: "0af5d786c11048bd901dcbe9aeea4498" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 35205 } }
I20260808 23:31:42.431773 5438 leader_election.cc:288] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26 [CANDIDATE]: Term 1 election: Requested vote from peers c7dcfeb49e8440f98e7957c68f3c376b (127.9.44.66:40357), 0af5d786c11048bd901dcbe9aeea4498 (127.9.44.67:35205)
I20260808 23:31:42.432219 5149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09dfcb4a9086489a8248f0ba38e36619" candidate_uuid: "3efef993c88448b98b55cff6318efc26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7dcfeb49e8440f98e7957c68f3c376b"
I20260808 23:31:42.432235 5340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09dfcb4a9086489a8248f0ba38e36619" candidate_uuid: "3efef993c88448b98b55cff6318efc26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0af5d786c11048bd901dcbe9aeea4498"
I20260808 23:31:42.432413 5340 raft_consensus.cc:3037] T 09dfcb4a9086489a8248f0ba38e36619 P 0af5d786c11048bd901dcbe9aeea4498 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:42.432413 5149 raft_consensus.cc:3037] T 09dfcb4a9086489a8248f0ba38e36619 P c7dcfeb49e8440f98e7957c68f3c376b [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:42.435478 5340 raft_consensus.cc:2454] T 09dfcb4a9086489a8248f0ba38e36619 P 0af5d786c11048bd901dcbe9aeea4498 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3efef993c88448b98b55cff6318efc26 in term 1.
I20260808 23:31:42.435478 5149 raft_consensus.cc:2454] T 09dfcb4a9086489a8248f0ba38e36619 P c7dcfeb49e8440f98e7957c68f3c376b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3efef993c88448b98b55cff6318efc26 in term 1.
I20260808 23:31:42.435869 4862 leader_election.cc:302] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0af5d786c11048bd901dcbe9aeea4498, 3efef993c88448b98b55cff6318efc26; no voters:
I20260808 23:31:42.436141 5438 raft_consensus.cc:2781] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:31:42.436537 5438 raft_consensus.cc:686] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26 [term 1 LEADER]: Becoming Leader. State: Replica: 3efef993c88448b98b55cff6318efc26, State: Running, Role: LEADER
I20260808 23:31:42.436959 5438 consensus_queue.cc:237] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3efef993c88448b98b55cff6318efc26" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42231 } } peers { permanent_uuid: "c7dcfeb49e8440f98e7957c68f3c376b" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 40357 } } peers { permanent_uuid: "0af5d786c11048bd901dcbe9aeea4498" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 35205 } }
I20260808 23:31:42.440999 4760 catalog_manager.cc:5495] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3efef993c88448b98b55cff6318efc26 (127.9.44.65). New cstate: current_term: 1 leader_uuid: "3efef993c88448b98b55cff6318efc26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3efef993c88448b98b55cff6318efc26" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7dcfeb49e8440f98e7957c68f3c376b" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0af5d786c11048bd901dcbe9aeea4498" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 35205 } health_report { overall_health: UNKNOWN } } }
I20260808 23:31:42.474902 5422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:42.469526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34564 (local address 127.9.44.65:42231)
0808 23:31:42.469725 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:42.469730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:42.469748 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:31:42.469926 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:42.469932 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:42.470003 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:42.470126 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:31:42.470523 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.471240 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.472648 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.473096 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.473202 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:42.473798 (+ 596us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:31:42.474312 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:31:42.474478 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:42.474634 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 23:31:42.474679 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260808 23:31:42.475994 5212 tablet.cc:2369] T 09dfcb4a9086489a8248f0ba38e36619 P c7dcfeb49e8440f98e7957c68f3c376b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:31:42.477800 5021 tablet.cc:2369] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:31:42.480662 5408 tablet.cc:2369] T 09dfcb4a9086489a8248f0ba38e36619 P 0af5d786c11048bd901dcbe9aeea4498: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:31:42.482613 5149 raft_consensus.cc:1261] T 09dfcb4a9086489a8248f0ba38e36619 P c7dcfeb49e8440f98e7957c68f3c376b [term 1 FOLLOWER]: Refusing update from remote peer 3efef993c88448b98b55cff6318efc26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:31:42.482843 5340 raft_consensus.cc:1261] T 09dfcb4a9086489a8248f0ba38e36619 P 0af5d786c11048bd901dcbe9aeea4498 [term 1 FOLLOWER]: Refusing update from remote peer 3efef993c88448b98b55cff6318efc26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:31:42.483584 5444 consensus_queue.cc:1035] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7dcfeb49e8440f98e7957c68f3c376b" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:31:42.483974 5438 consensus_queue.cc:1035] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0af5d786c11048bd901dcbe9aeea4498" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 35205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:31:42.496898 5451 mvcc.cc:205] Tried to move back new op lower bound from 7316405872559464448 to 7316405872385159168. Current Snapshot: MvccSnapshot[applied={T|T < 7316405872559464448}]
I20260808 23:31:42.506991 4759 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:31:42.507258 4759 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:31:42.510514 4759 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ff9a46b29e24b8687902547ea8ba661: Sending DeleteTablet for 3 replicas of tablet 09dfcb4a9086489a8248f0ba38e36619
I20260808 23:31:42.511548 4928 tablet_service.cc:1514] Processing DeleteTablet for tablet 09dfcb4a9086489a8248f0ba38e36619 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:31:42 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:34550
I20260808 23:31:42.511550 5121 tablet_service.cc:1514] Processing DeleteTablet for tablet 09dfcb4a9086489a8248f0ba38e36619 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:31:42 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:58884
I20260808 23:31:42.511806 5309 tablet_service.cc:1514] Processing DeleteTablet for tablet 09dfcb4a9086489a8248f0ba38e36619 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:31:42 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:34458
I20260808 23:31:42.515547 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 4844
I20260808 23:31:42.523823 5459 tablet_replica.cc:330] T 09dfcb4a9086489a8248f0ba38e36619 P c7dcfeb49e8440f98e7957c68f3c376b: stopping tablet replica
I20260808 23:31:42.524052 5460 tablet_replica.cc:330] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26: stopping tablet replica
I20260808 23:31:42.524216 5459 raft_consensus.cc:2229] T 09dfcb4a9086489a8248f0ba38e36619 P c7dcfeb49e8440f98e7957c68f3c376b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:31:42.524403 5460 raft_consensus.cc:2229] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:31:42.524683 5459 raft_consensus.cc:2258] T 09dfcb4a9086489a8248f0ba38e36619 P c7dcfeb49e8440f98e7957c68f3c376b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:31:42.525053 5460 raft_consensus.cc:2258] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:31:42.525722 5462 tablet_replica.cc:330] T 09dfcb4a9086489a8248f0ba38e36619 P 0af5d786c11048bd901dcbe9aeea4498: stopping tablet replica
I20260808 23:31:42.526196 5462 raft_consensus.cc:2229] T 09dfcb4a9086489a8248f0ba38e36619 P 0af5d786c11048bd901dcbe9aeea4498 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:31:42.526790 5462 raft_consensus.cc:2258] T 09dfcb4a9086489a8248f0ba38e36619 P 0af5d786c11048bd901dcbe9aeea4498 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:31:42.526790 5459 ts_tablet_manager.cc:1905] T 09dfcb4a9086489a8248f0ba38e36619 P c7dcfeb49e8440f98e7957c68f3c376b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:31:42.528808 5462 ts_tablet_manager.cc:1905] T 09dfcb4a9086489a8248f0ba38e36619 P 0af5d786c11048bd901dcbe9aeea4498: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:31:42.533905 5459 ts_tablet_manager.cc:1918] T 09dfcb4a9086489a8248f0ba38e36619 P c7dcfeb49e8440f98e7957c68f3c376b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:31:42.534129 5459 log.cc:1196] T 09dfcb4a9086489a8248f0ba38e36619 P c7dcfeb49e8440f98e7957c68f3c376b: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/09dfcb4a9086489a8248f0ba38e36619
I20260808 23:31:42.534726 5459 ts_tablet_manager.cc:1939] T 09dfcb4a9086489a8248f0ba38e36619 P c7dcfeb49e8440f98e7957c68f3c376b: Deleting consensus metadata
I20260808 23:31:42.535784 5462 ts_tablet_manager.cc:1918] T 09dfcb4a9086489a8248f0ba38e36619 P 0af5d786c11048bd901dcbe9aeea4498: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:31:42.536531 4742 catalog_manager.cc:4841] TS c7dcfeb49e8440f98e7957c68f3c376b (127.9.44.66:40357): tablet 09dfcb4a9086489a8248f0ba38e36619 (table test-table [id=da7d3b7da0144a6ca4daa7259141c23f]) successfully deleted
I20260808 23:31:42.537155 5460 ts_tablet_manager.cc:1905] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:31:42.537580 5462 log.cc:1196] T 09dfcb4a9086489a8248f0ba38e36619 P 0af5d786c11048bd901dcbe9aeea4498: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/09dfcb4a9086489a8248f0ba38e36619
I20260808 23:31:42.538144 5462 ts_tablet_manager.cc:1939] T 09dfcb4a9086489a8248f0ba38e36619 P 0af5d786c11048bd901dcbe9aeea4498: Deleting consensus metadata
I20260808 23:31:42.539779 4740 catalog_manager.cc:4841] TS 0af5d786c11048bd901dcbe9aeea4498 (127.9.44.67:35205): tablet 09dfcb4a9086489a8248f0ba38e36619 (table test-table [id=da7d3b7da0144a6ca4daa7259141c23f]) successfully deleted
I20260808 23:31:42.542346 5460 ts_tablet_manager.cc:1918] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:31:42.542500 5460 log.cc:1196] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786231597224934-9393-0/minicluster-data/ts-0/wal/wals/09dfcb4a9086489a8248f0ba38e36619
I20260808 23:31:42.543031 5460 ts_tablet_manager.cc:1939] T 09dfcb4a9086489a8248f0ba38e36619 P 3efef993c88448b98b55cff6318efc26: Deleting consensus metadata
Aug 08 23:31:42 dist-test-slave-6bbx krb5kdc[4701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.65@KRBTEST.COM
I20260808 23:31:42.544618 4743 catalog_manager.cc:4841] TS 3efef993c88448b98b55cff6318efc26 (127.9.44.65:42231): tablet 09dfcb4a9086489a8248f0ba38e36619 (table test-table [id=da7d3b7da0144a6ca4daa7259141c23f]) successfully deleted
I20260808 23:31:42.551513 5422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:42.516935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34572 (local address 127.9.44.65:42231)
0808 23:31:42.517115 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:42.517122 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:42.530503 (+ 13381us) server_negotiation.cc:407] Connection header received
0808 23:31:42.530922 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:42.530928 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:42.531003 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:42.531072 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:42.532873 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.533982 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.535881 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.536601 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.536699 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:42.545068 (+ 8369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:42.545082 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:42.545102 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:42.545140 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:42.548805 (+ 3665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:42.549444 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:42.549452 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:42.549456 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:42.549636 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:42.549973 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:42.549978 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:42.549981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:42.550442 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:42.550746 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:42.550911 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 23:31:42.551137 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 23:31:42.552135 5014 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:42.648352 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 4844
I20260808 23:31:42.672402 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 5027
Aug 08 23:31:42 dist-test-slave-6bbx krb5kdc[4701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.66@KRBTEST.COM
I20260808 23:31:42.689078 5425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:42.673131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58886 (local address 127.9.44.66:40357)
0808 23:31:42.673319 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:42.673324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:42.673342 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:31:42.673520 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:42.673524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:42.673596 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:42.673673 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:42.674912 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.675615 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.677038 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.677417 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.677508 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:42.682486 (+ 4978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:42.682504 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:42.682525 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:42.682572 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:42.686092 (+ 3520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:42.686967 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:42.686975 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:42.686978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:42.687147 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:42.687502 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:42.687508 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:42.687511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:42.688102 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:42.688441 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:42.688608 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 23:31:42.688760 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 23:31:42.689954 5204 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:42.788022 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 5027
I20260808 23:31:42.807031 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 5220
Aug 08 23:31:42 dist-test-slave-6bbx krb5kdc[4701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.67@KRBTEST.COM
I20260808 23:31:42.820849 5427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:42.807767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34474 (local address 127.9.44.67:35205)
0808 23:31:42.807923 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:42.807928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:42.807945 (+ 17us) server_negotiation.cc:407] Connection header received
0808 23:31:42.808058 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:42.808062 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:42.808128 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:42.808207 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:42.809442 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.810136 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.811279 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:42.811652 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:42.811774 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:42.815723 (+ 3949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:42.815740 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:42.815756 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:42.815791 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:42.818288 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:42.818989 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:42.818996 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:42.818998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:42.819136 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:42.819417 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:42.819420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:42.819422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:42.819974 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:42.820193 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:42.820442 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 23:31:42.820606 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 23:31:42.821458 5399 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:42.909516 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 5220
I20260808 23:31:42.930172 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 4720
I20260808 23:31:42.931216 4802 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:43.037138 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 4720
2026-08-08T23:31:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4294 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T23:31:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:31:43Z Disabled control of system clock
I20260808 23:31:43.125782 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:33939
--webserver_interface=127.9.44.126
--webserver_port=0
--builtin_ntp_servers=127.9.44.84:39315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:33939
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 23:31:43.444482 5475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:43.444761 5475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:43.444891 5475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:43.451687 5475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:31:43.451785 5475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:43.451857 5475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:43.451936 5475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:31:43.451993 5475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:31:43.460618 5475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:39315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:33939
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:33939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:43.461748 5475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:43.463392 5475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:43.474124 5482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:43.474352 5481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:43.474370 5484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:43.475109 5475 server_base.cc:1034] running on GCE node
I20260808 23:31:43.475895 5475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:43.477160 5475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:43.478358 5475 hybrid_clock.cc:648] HybridClock initialized: now 1786231903478315 us; error 35 us; skew 500 ppm
I20260808 23:31:43.478665 5475 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:31:43.480769 5475 webserver.cc:460] Webserver started at http://127.9.44.126:36017/ using document root <none> and password file <none>
I20260808 23:31:43.481395 5475 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:43.481492 5475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:43.481784 5475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:43.484627 5475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "9cc36b10a1004b3a8d6d684508b0fb6e"
format_stamp: "Formatted at 2026-08-08 23:31:43 on dist-test-slave-6bbx"
server_key: "62fab178fe4b81bd98acc2cd9a1a0979"
server_key_iv: "58c9966246011c11c19272365e2a96bb"
server_key_version: "encryptionkey@0"
I20260808 23:31:43.485390 5475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "9cc36b10a1004b3a8d6d684508b0fb6e"
format_stamp: "Formatted at 2026-08-08 23:31:43 on dist-test-slave-6bbx"
server_key: "62fab178fe4b81bd98acc2cd9a1a0979"
server_key_iv: "58c9966246011c11c19272365e2a96bb"
server_key_version: "encryptionkey@0"
I20260808 23:31:43.490062 5475 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 23:31:43.493238 5491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:43.494407 5475 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 23:31:43.494558 5475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "9cc36b10a1004b3a8d6d684508b0fb6e"
format_stamp: "Formatted at 2026-08-08 23:31:43 on dist-test-slave-6bbx"
server_key: "62fab178fe4b81bd98acc2cd9a1a0979"
server_key_iv: "58c9966246011c11c19272365e2a96bb"
server_key_version: "encryptionkey@0"
I20260808 23:31:43.494740 5475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:43.517475 5475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:43.518180 5475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:43.518429 5475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:43.534768 5475 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:33939
I20260808 23:31:43.534788 5561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:33939 every 8 connection(s)
I20260808 23:31:43.536010 5475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:31:43.539448 5562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:43.546330 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 5475
I20260808 23:31:43.546517 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:31:43.547099 9393 external_mini_cluster.cc:1428] Setting key 48d09b52d461ab97b286e8e7b0302353
I20260808 23:31:43.550035 5562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cc36b10a1004b3a8d6d684508b0fb6e: Bootstrap starting.
I20260808 23:31:43.553275 5562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9cc36b10a1004b3a8d6d684508b0fb6e: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:43.554297 5562 log.cc:824] T 00000000000000000000000000000000 P 9cc36b10a1004b3a8d6d684508b0fb6e: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:43.557183 5562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cc36b10a1004b3a8d6d684508b0fb6e: No bootstrap required, opened a new log
I20260808 23:31:43.559777 5565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:43.548900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43862 (local address 127.9.44.126:33939)
0808 23:31:43.549777 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:43.549787 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:43.549826 (+ 39us) server_negotiation.cc:407] Connection header received
0808 23:31:43.551520 (+ 1694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:43.551541 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:43.551986 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:43.552272 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:43.552848 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:43.554128 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:43.555742 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:43.556379 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:43.556508 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:43.557674 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:43.557681 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:43.557694 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:43.558179 (+ 485us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:43.558250 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:43.558270 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:43.558361 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:43.558601 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 23:31:43.558874 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":309,"threads_started":1}
I20260808 23:31:43.564605 5562 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9cc36b10a1004b3a8d6d684508b0fb6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc36b10a1004b3a8d6d684508b0fb6e" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 33939 } }
I20260808 23:31:43.564926 5562 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9cc36b10a1004b3a8d6d684508b0fb6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:43.565018 5562 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9cc36b10a1004b3a8d6d684508b0fb6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cc36b10a1004b3a8d6d684508b0fb6e, State: Initialized, Role: FOLLOWER
I20260808 23:31:43.565523 5562 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9cc36b10a1004b3a8d6d684508b0fb6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc36b10a1004b3a8d6d684508b0fb6e" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 33939 } }
I20260808 23:31:43.565749 5562 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9cc36b10a1004b3a8d6d684508b0fb6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:31:43.565877 5562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9cc36b10a1004b3a8d6d684508b0fb6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:31:43.566025 5562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9cc36b10a1004b3a8d6d684508b0fb6e [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:43.568262 5562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9cc36b10a1004b3a8d6d684508b0fb6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc36b10a1004b3a8d6d684508b0fb6e" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 33939 } }
I20260808 23:31:43.568776 5562 leader_election.cc:302] T 00000000000000000000000000000000 P 9cc36b10a1004b3a8d6d684508b0fb6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9cc36b10a1004b3a8d6d684508b0fb6e; no voters:
I20260808 23:31:43.569299 5562 leader_election.cc:288] T 00000000000000000000000000000000 P 9cc36b10a1004b3a8d6d684508b0fb6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:31:43.569417 5568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9cc36b10a1004b3a8d6d684508b0fb6e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:31:43.570039 5568 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9cc36b10a1004b3a8d6d684508b0fb6e [term 1 LEADER]: Becoming Leader. State: Replica: 9cc36b10a1004b3a8d6d684508b0fb6e, State: Running, Role: LEADER
I20260808 23:31:43.570565 5568 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9cc36b10a1004b3a8d6d684508b0fb6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc36b10a1004b3a8d6d684508b0fb6e" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 33939 } }
I20260808 23:31:43.571146 5562 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9cc36b10a1004b3a8d6d684508b0fb6e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:31:43.573026 5569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cc36b10a1004b3a8d6d684508b0fb6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9cc36b10a1004b3a8d6d684508b0fb6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc36b10a1004b3a8d6d684508b0fb6e" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 33939 } } }
I20260808 23:31:43.573035 5570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cc36b10a1004b3a8d6d684508b0fb6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9cc36b10a1004b3a8d6d684508b0fb6e. Latest consensus state: current_term: 1 leader_uuid: "9cc36b10a1004b3a8d6d684508b0fb6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc36b10a1004b3a8d6d684508b0fb6e" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 33939 } } }
I20260808 23:31:43.573627 5570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cc36b10a1004b3a8d6d684508b0fb6e [sys.catalog]: This master's current role is: LEADER
I20260808 23:31:43.573628 5569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cc36b10a1004b3a8d6d684508b0fb6e [sys.catalog]: This master's current role is: LEADER
I20260808 23:31:43.574414 5576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:31:43.579658 5576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:31:43.590137 5576 catalog_manager.cc:1347] Generated new cluster ID: dc0be82af9d44491b91f83b0593aacc7
I20260808 23:31:43.590296 5576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:31:43.602241 5576 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:31:43.603433 5576 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:31:43.618942 5576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9cc36b10a1004b3a8d6d684508b0fb6e: Generated new TSK 0
I20260808 23:31:43.619683 5576 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:31:43.630569 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:0
--local_ip_for_outbound_sockets=127.9.44.65
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:33939
--builtin_ntp_servers=127.9.44.84:39315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 23:31:43.956287 5593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:43.956532 5593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:43.956651 5593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:43.963516 5593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:43.963606 5593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:43.963764 5593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:31:43.972402 5593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:39315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:33939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:43.973501 5593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:43.975144 5593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:43.987442 5601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:43.987491 5602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:43.987529 5606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:43.988415 5593 server_base.cc:1034] running on GCE node
I20260808 23:31:43.988930 5593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:43.989751 5593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:43.990938 5593 hybrid_clock.cc:648] HybridClock initialized: now 1786231903990892 us; error 33 us; skew 500 ppm
I20260808 23:31:43.991214 5593 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:31:43.993306 5593 webserver.cc:460] Webserver started at http://127.9.44.65:33741/ using document root <none> and password file <none>
I20260808 23:31:43.993922 5593 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:43.994014 5593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:43.994308 5593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:43.996922 5593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-0/data/instance:
uuid: "987081c49c244dc085ecade0ccc398ea"
format_stamp: "Formatted at 2026-08-08 23:31:43 on dist-test-slave-6bbx"
server_key: "29434c87bd7f66055fe64539495512ff"
server_key_iv: "46887c0fca19c0c461f328ae099443c7"
server_key_version: "encryptionkey@0"
I20260808 23:31:43.997709 5593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance:
uuid: "987081c49c244dc085ecade0ccc398ea"
format_stamp: "Formatted at 2026-08-08 23:31:43 on dist-test-slave-6bbx"
server_key: "29434c87bd7f66055fe64539495512ff"
server_key_iv: "46887c0fca19c0c461f328ae099443c7"
server_key_version: "encryptionkey@0"
I20260808 23:31:44.002451 5593 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 23:31:44.005621 5616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:44.006770 5593 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 23:31:44.006912 5593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "987081c49c244dc085ecade0ccc398ea"
format_stamp: "Formatted at 2026-08-08 23:31:43 on dist-test-slave-6bbx"
server_key: "29434c87bd7f66055fe64539495512ff"
server_key_iv: "46887c0fca19c0c461f328ae099443c7"
server_key_version: "encryptionkey@0"
I20260808 23:31:44.007094 5593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:44.027004 5593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:44.027698 5593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:44.027936 5593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:44.028776 5593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:31:44.030081 5593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:31:44.030167 5593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:44.030261 5593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:31:44.030347 5593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:44.059232 5593 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:40687
I20260808 23:31:44.059336 5781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:40687 every 8 connection(s)
I20260808 23:31:44.060488 5593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:31:44.069372 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 5593
I20260808 23:31:44.069516 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance
I20260808 23:31:44.069929 9393 external_mini_cluster.cc:1428] Setting key 036966ad97554c2f75cc6f13637f38d5
I20260808 23:31:44.072573 5784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:44.062797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:46651 (local address 127.9.44.126:33939)
0808 23:31:44.063234 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:44.063238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:44.064332 (+ 1094us) server_negotiation.cc:407] Connection header received
0808 23:31:44.066070 (+ 1738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:44.066075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:44.066159 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:44.066257 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:44.067280 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:44.068060 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:44.069591 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:44.070117 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:44.070232 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:44.071617 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:44.071623 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:44.071627 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:44.071649 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:44.071732 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:44.071751 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:44.071842 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:44.072088 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 23:31:44.072248 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":219,"threads_started":1}
I20260808 23:31:44.072799 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:0
--local_ip_for_outbound_sockets=127.9.44.66
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:33939
--builtin_ntp_servers=127.9.44.84:39315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 23:31:44.073644 5785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:44.063321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:33939 (local address 127.9.44.65:46651)
0808 23:31:44.064190 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:44.064225 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:44.065800 (+ 1575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:44.066500 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:44.066509 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:44.066591 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:31:44.067113 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:44.067129 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:44.068214 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:44.068218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:44.069420 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:44.069436 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:44.070352 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:44.070360 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:44.070600 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:44.071286 (+ 686us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:31:44.071289 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:31:44.071335 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:31:44.071408 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:31:44.071850 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:31:44.071861 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:31:44.071869 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 23:31:44.072078 (+ 209us) client_negotiation.cc:240] Negotiation successful
0808 23:31:44.072493 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":284,"threads_started":1}
I20260808 23:31:44.075116 5782 heartbeater.cc:347] Connected to a master server at 127.9.44.126:33939
I20260808 23:31:44.075402 5782 heartbeater.cc:464] Registering TS with master...
I20260808 23:31:44.076150 5782 heartbeater.cc:511] Master 127.9.44.126:33939 requested a full tablet report, sending...
I20260808 23:31:44.078047 5510 ts_manager.cc:194] Registered new tserver with Master: 987081c49c244dc085ecade0ccc398ea (127.9.44.65:40687)
I20260808 23:31:44.079466 5510 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.44.65:46651
W20260808 23:31:44.392472 5787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:44.392717 5787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:44.392836 5787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:44.399755 5787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:44.399850 5787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:44.400002 5787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:31:44.408665 5787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:39315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:33939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:44.409787 5787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:44.411516 5787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:44.423477 5797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:44.423508 5793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:44.423434 5794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:44.424263 5787 server_base.cc:1034] running on GCE node
I20260808 23:31:44.424825 5787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:44.425693 5787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:44.426900 5787 hybrid_clock.cc:648] HybridClock initialized: now 1786231904426857 us; error 31 us; skew 500 ppm
I20260808 23:31:44.427192 5787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:31:44.429262 5787 webserver.cc:460] Webserver started at http://127.9.44.66:40661/ using document root <none> and password file <none>
I20260808 23:31:44.429904 5787 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:44.430007 5787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:44.430346 5787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:44.433120 5787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-1/data/instance:
uuid: "25522c36305a47608578bc6b1fd3818f"
format_stamp: "Formatted at 2026-08-08 23:31:44 on dist-test-slave-6bbx"
server_key: "ee3c9380fc64cfe2d61aa02db4b0fb97"
server_key_iv: "06e8da261c7eb21f25114dccd29abe93"
server_key_version: "encryptionkey@0"
I20260808 23:31:44.433902 5787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance:
uuid: "25522c36305a47608578bc6b1fd3818f"
format_stamp: "Formatted at 2026-08-08 23:31:44 on dist-test-slave-6bbx"
server_key: "ee3c9380fc64cfe2d61aa02db4b0fb97"
server_key_iv: "06e8da261c7eb21f25114dccd29abe93"
server_key_version: "encryptionkey@0"
I20260808 23:31:44.438606 5787 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260808 23:31:44.441883 5803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:44.443070 5787 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 23:31:44.443219 5787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "25522c36305a47608578bc6b1fd3818f"
format_stamp: "Formatted at 2026-08-08 23:31:44 on dist-test-slave-6bbx"
server_key: "ee3c9380fc64cfe2d61aa02db4b0fb97"
server_key_iv: "06e8da261c7eb21f25114dccd29abe93"
server_key_version: "encryptionkey@0"
I20260808 23:31:44.443414 5787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:44.464427 5787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:44.465329 5787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:44.465617 5787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:44.466739 5787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:31:44.468528 5787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:31:44.468641 5787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:44.468742 5787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:31:44.468815 5787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:44.499693 5787 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:36433
I20260808 23:31:44.499770 5961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:36433 every 8 connection(s)
I20260808 23:31:44.501257 5787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:31:44.514394 5784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:44.503875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:44459 (local address 127.9.44.126:33939)
0808 23:31:44.504043 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:44.504048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:44.505770 (+ 1722us) server_negotiation.cc:407] Connection header received
0808 23:31:44.507521 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:44.507527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:44.507608 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:44.507722 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:44.508725 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:44.509454 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:44.511144 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:44.511811 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:44.511926 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:44.513550 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:44.513557 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:44.513562 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:44.513582 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:44.513619 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:44.513633 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:44.513686 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:44.513978 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 23:31:44.514076 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 23:31:44.516301 5965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:44.504600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:33939 (local address 127.9.44.66:44459)
0808 23:31:44.505620 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:44.505663 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:44.507255 (+ 1592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:44.507937 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:44.507946 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:44.508026 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:31:44.508561 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:44.508578 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:44.509650 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:44.509654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:44.511004 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:44.511021 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:44.512215 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:44.512221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:44.512508 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:44.513196 (+ 688us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:31:44.513199 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:31:44.513248 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:31:44.513350 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:31:44.513761 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:31:44.513770 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:31:44.513777 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 23:31:44.514802 (+ 1025us) client_negotiation.cc:240] Negotiation successful
0808 23:31:44.515151 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":390,"threads_started":1}
I20260808 23:31:44.517869 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 5787
I20260808 23:31:44.518036 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance
I20260808 23:31:44.518409 5962 heartbeater.cc:347] Connected to a master server at 127.9.44.126:33939
I20260808 23:31:44.518568 9393 external_mini_cluster.cc:1428] Setting key c416b9aad64ee5c8fc308a079e9ad1bd
I20260808 23:31:44.518764 5962 heartbeater.cc:464] Registering TS with master...
I20260808 23:31:44.519737 5962 heartbeater.cc:511] Master 127.9.44.126:33939 requested a full tablet report, sending...
I20260808 23:31:44.521347 5510 ts_manager.cc:194] Registered new tserver with Master: 25522c36305a47608578bc6b1fd3818f (127.9.44.66:36433)
I20260808 23:31:44.521421 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:0
--local_ip_for_outbound_sockets=127.9.44.67
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:33939
--builtin_ntp_servers=127.9.44.84:39315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 23:31:44.522361 5510 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.44.66:44459
W20260808 23:31:44.863703 5967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:44.863958 5967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:44.864077 5967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:44.870940 5967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:44.871029 5967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:44.871191 5967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:31:44.880081 5967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:39315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:33939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:44.881165 5967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:44.882994 5967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:44.895643 5979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:44.895744 5976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:44.895716 5977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:44.897279 5967 server_base.cc:1034] running on GCE node
I20260808 23:31:44.897971 5967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:44.899359 5967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:44.900569 5967 hybrid_clock.cc:648] HybridClock initialized: now 1786231904900516 us; error 46 us; skew 500 ppm
I20260808 23:31:44.900882 5967 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:31:44.903002 5967 webserver.cc:460] Webserver started at http://127.9.44.67:44637/ using document root <none> and password file <none>
I20260808 23:31:44.903651 5967 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:44.903792 5967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:44.904114 5967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:44.906986 5967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-2/data/instance:
uuid: "716e8cc9acc24153892296893268ae65"
format_stamp: "Formatted at 2026-08-08 23:31:44 on dist-test-slave-6bbx"
server_key: "1edaa63c669dbe214880ce30e3de434d"
server_key_iv: "ebed7512c751497bf5f5bf7ac7c11c0d"
server_key_version: "encryptionkey@0"
I20260808 23:31:44.907785 5967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance:
uuid: "716e8cc9acc24153892296893268ae65"
format_stamp: "Formatted at 2026-08-08 23:31:44 on dist-test-slave-6bbx"
server_key: "1edaa63c669dbe214880ce30e3de434d"
server_key_iv: "ebed7512c751497bf5f5bf7ac7c11c0d"
server_key_version: "encryptionkey@0"
I20260808 23:31:44.912803 5967 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 23:31:44.916172 5985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:44.917413 5967 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 23:31:44.917567 5967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "716e8cc9acc24153892296893268ae65"
format_stamp: "Formatted at 2026-08-08 23:31:44 on dist-test-slave-6bbx"
server_key: "1edaa63c669dbe214880ce30e3de434d"
server_key_iv: "ebed7512c751497bf5f5bf7ac7c11c0d"
server_key_version: "encryptionkey@0"
I20260808 23:31:44.917764 5967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:44.935873 5967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:44.936630 5967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:44.936908 5967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:44.937808 5967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:31:44.939235 5967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:31:44.939333 5967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:44.939440 5967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:31:44.939517 5967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:44.970135 5967 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:44425
I20260808 23:31:44.970191 6142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:44425 every 8 connection(s)
I20260808 23:31:44.971457 5967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:31:44.976758 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 5967
I20260808 23:31:44.976934 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance
I20260808 23:31:44.977419 9393 external_mini_cluster.cc:1428] Setting key 34f08c164cb7940b62aae41ac9f46967
I20260808 23:31:44.984161 5784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:44.974069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:36613 (local address 127.9.44.126:33939)
0808 23:31:44.974245 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:44.974250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:44.975596 (+ 1346us) server_negotiation.cc:407] Connection header received
0808 23:31:44.977544 (+ 1948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:44.977550 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:44.977637 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:44.977736 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:44.978967 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:44.980004 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:44.981556 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:44.981967 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:44.982073 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:44.983313 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:44.983318 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:44.983321 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:44.983338 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:44.983369 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:44.983380 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:44.983451 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:44.983691 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 23:31:44.983799 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 23:31:44.985083 6146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:44.974538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:33939 (local address 127.9.44.67:36613)
0808 23:31:44.975421 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:44.975479 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:44.977286 (+ 1807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:44.977967 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:44.977976 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:44.978081 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:31:44.978793 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:44.978812 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:44.980167 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:44.980173 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:44.981414 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:44.981427 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:44.982190 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:44.982199 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:44.982427 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:44.983015 (+ 588us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:31:44.983018 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:31:44.983060 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:31:44.983136 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:31:44.983465 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:31:44.983478 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:31:44.983485 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 23:31:44.983659 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 23:31:44.984047 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":279,"threads_started":1}
I20260808 23:31:44.986395 6143 heartbeater.cc:347] Connected to a master server at 127.9.44.126:33939
I20260808 23:31:44.986652 6143 heartbeater.cc:464] Registering TS with master...
I20260808 23:31:44.987381 6143 heartbeater.cc:511] Master 127.9.44.126:33939 requested a full tablet report, sending...
I20260808 23:31:44.988718 5510 ts_manager.cc:194] Registered new tserver with Master: 716e8cc9acc24153892296893268ae65 (127.9.44.67:44425)
I20260808 23:31:44.989496 5510 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.44.67:36613
I20260808 23:31:44.992410 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:31:45.002305 5784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:44.996581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43864 (local address 127.9.44.126:33939)
0808 23:31:44.996729 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:44.996733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:44.997195 (+ 462us) server_negotiation.cc:407] Connection header received
0808 23:31:44.997359 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:44.997364 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:44.997427 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:44.997508 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:44.998034 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:44.998729 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.000151 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.000559 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.000649 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:45.001612 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:45.001617 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:45.001620 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:45.001637 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:45.001668 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:45.001679 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:45.001782 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:45.001958 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 23:31:45.002003 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 23:31:45.005903 5510 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43864:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:31:45.013095 5510 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:31:45.046144 6162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:45.024978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48152 (local address 127.9.44.65:40687)
0808 23:31:45.036111 (+ 11133us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:45.036121 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:45.036987 (+ 866us) server_negotiation.cc:407] Connection header received
0808 23:31:45.037258 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:45.037267 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:45.037541 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:45.037735 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:45.038667 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.040254 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.043572 (+ 3318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.045066 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.045217 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:45.045456 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:45.045726 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 23:31:45.045858 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10972,"thread_start_us":10847,"threads_started":1}
I20260808 23:31:45.046270 6161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:45.024910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:40687 (local address 127.0.0.1:48152)
0808 23:31:45.036814 (+ 11904us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:45.036856 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:45.037038 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:45.037989 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:45.037997 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:45.038025 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:45.038439 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:45.038453 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.040411 (+ 1958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.040416 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:45.042946 (+ 2530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:45.042962 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.045279 (+ 2317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.045286 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:45.045496 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:45.045548 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 23:31:45.045698 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 23:31:45.045916 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11687,"mutex_wait_us":4,"thread_start_us":12378,"threads_started":1}
I20260808 23:31:45.047729 6164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:45.025361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43302 (local address 127.9.44.66:36433)
0808 23:31:45.036391 (+ 11030us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:45.036399 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:45.037868 (+ 1469us) server_negotiation.cc:407] Connection header received
0808 23:31:45.037975 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:45.037983 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:45.038210 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:45.038413 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:45.039254 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.040724 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.043630 (+ 2906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.045066 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.045217 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:45.045465 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:45.047207 (+ 1742us) server_negotiation.cc:299] Negotiation successful
0808 23:31:45.047384 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10887,"thread_start_us":10795,"threads_started":1}
I20260808 23:31:45.048493 6160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:45.024968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:36433 (local address 127.0.0.1:43302)
0808 23:31:45.037585 (+ 12617us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:45.037608 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:45.037741 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:45.038658 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:45.038662 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:45.038688 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:45.039064 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:45.039076 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.040895 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.040904 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:45.043338 (+ 2434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:45.043352 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.046803 (+ 3451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.046812 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:45.046990 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:45.047021 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 23:31:45.047973 (+ 952us) client_negotiation.cc:240] Negotiation successful
0808 23:31:45.048098 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12482,"mutex_wait_us":259,"thread_start_us":11548,"threads_started":1}
I20260808 23:31:45.050222 5687 tablet_service.cc:1467] Processing CreateTablet for tablet 6deefd2c8e56432cbf3ca5310f3c143e (DEFAULT_TABLE table=test-table [id=d199979c3efb476ba83b30cc4a61defc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:31:45.051898 5687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6deefd2c8e56432cbf3ca5310f3c143e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:45.052572 6166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:45.037658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:44425 (local address 127.0.0.1:60578)
0808 23:31:45.043286 (+ 5628us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:45.043310 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:45.043422 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:45.045533 (+ 2111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:45.045538 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:45.045563 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:45.045975 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:45.045987 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.048131 (+ 2144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.048135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:45.049918 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:45.049931 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.051880 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.051887 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:45.052055 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:45.052087 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 23:31:45.052181 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 23:31:45.052299 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5501,"thread_start_us":5377,"threads_started":1}
I20260808 23:31:45.052210 5873 tablet_service.cc:1467] Processing CreateTablet for tablet 6deefd2c8e56432cbf3ca5310f3c143e (DEFAULT_TABLE table=test-table [id=d199979c3efb476ba83b30cc4a61defc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:31:45.052753 6167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:45.037763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60578 (local address 127.9.44.67:44425)
0808 23:31:45.044797 (+ 7034us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:45.044802 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:45.044828 (+ 26us) server_negotiation.cc:407] Connection header received
0808 23:31:45.044902 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:45.044907 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:45.045076 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:45.045257 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:45.046247 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.047975 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.050353 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.051696 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.051828 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:45.052099 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:45.052225 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 23:31:45.052395 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6881,"thread_start_us":6727,"threads_started":1}
I20260808 23:31:45.053840 5873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6deefd2c8e56432cbf3ca5310f3c143e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:45.055845 6050 tablet_service.cc:1467] Processing CreateTablet for tablet 6deefd2c8e56432cbf3ca5310f3c143e (DEFAULT_TABLE table=test-table [id=d199979c3efb476ba83b30cc4a61defc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:31:45.056948 6050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6deefd2c8e56432cbf3ca5310f3c143e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:45.072372 6169 tablet_bootstrap.cc:492] T 6deefd2c8e56432cbf3ca5310f3c143e P 987081c49c244dc085ecade0ccc398ea: Bootstrap starting.
I20260808 23:31:45.073360 6170 tablet_bootstrap.cc:492] T 6deefd2c8e56432cbf3ca5310f3c143e P 25522c36305a47608578bc6b1fd3818f: Bootstrap starting.
I20260808 23:31:45.079193 6170 tablet_bootstrap.cc:654] T 6deefd2c8e56432cbf3ca5310f3c143e P 25522c36305a47608578bc6b1fd3818f: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:45.080649 6170 log.cc:824] T 6deefd2c8e56432cbf3ca5310f3c143e P 25522c36305a47608578bc6b1fd3818f: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:45.082172 5782 heartbeater.cc:503] Master 127.9.44.126:33939 was elected leader, sending a full tablet report...
I20260808 23:31:45.084367 6173 tablet_bootstrap.cc:492] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65: Bootstrap starting.
I20260808 23:31:45.086118 6169 tablet_bootstrap.cc:654] T 6deefd2c8e56432cbf3ca5310f3c143e P 987081c49c244dc085ecade0ccc398ea: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:45.087221 6169 log.cc:824] T 6deefd2c8e56432cbf3ca5310f3c143e P 987081c49c244dc085ecade0ccc398ea: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:45.090314 6173 tablet_bootstrap.cc:654] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:45.091429 6169 tablet_bootstrap.cc:492] T 6deefd2c8e56432cbf3ca5310f3c143e P 987081c49c244dc085ecade0ccc398ea: No bootstrap required, opened a new log
I20260808 23:31:45.091581 6173 log.cc:824] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:45.091825 6169 ts_tablet_manager.cc:1397] T 6deefd2c8e56432cbf3ca5310f3c143e P 987081c49c244dc085ecade0ccc398ea: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.005s
I20260808 23:31:45.095153 6173 tablet_bootstrap.cc:492] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65: No bootstrap required, opened a new log
I20260808 23:31:45.095355 6170 tablet_bootstrap.cc:492] T 6deefd2c8e56432cbf3ca5310f3c143e P 25522c36305a47608578bc6b1fd3818f: No bootstrap required, opened a new log
I20260808 23:31:45.095432 6173 ts_tablet_manager.cc:1397] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.002s
I20260808 23:31:45.095806 6170 ts_tablet_manager.cc:1397] T 6deefd2c8e56432cbf3ca5310f3c143e P 25522c36305a47608578bc6b1fd3818f: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.007s
I20260808 23:31:45.102435 6169 raft_consensus.cc:348] T 6deefd2c8e56432cbf3ca5310f3c143e P 987081c49c244dc085ecade0ccc398ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "987081c49c244dc085ecade0ccc398ea" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40687 } } peers { permanent_uuid: "25522c36305a47608578bc6b1fd3818f" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 36433 } } peers { permanent_uuid: "716e8cc9acc24153892296893268ae65" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44425 } }
I20260808 23:31:45.102897 6169 raft_consensus.cc:374] T 6deefd2c8e56432cbf3ca5310f3c143e P 987081c49c244dc085ecade0ccc398ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:45.103045 6169 raft_consensus.cc:729] T 6deefd2c8e56432cbf3ca5310f3c143e P 987081c49c244dc085ecade0ccc398ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 987081c49c244dc085ecade0ccc398ea, State: Initialized, Role: FOLLOWER
I20260808 23:31:45.103744 6169 consensus_queue.cc:260] T 6deefd2c8e56432cbf3ca5310f3c143e P 987081c49c244dc085ecade0ccc398ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "987081c49c244dc085ecade0ccc398ea" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40687 } } peers { permanent_uuid: "25522c36305a47608578bc6b1fd3818f" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 36433 } } peers { permanent_uuid: "716e8cc9acc24153892296893268ae65" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44425 } }
I20260808 23:31:45.104832 6170 raft_consensus.cc:348] T 6deefd2c8e56432cbf3ca5310f3c143e P 25522c36305a47608578bc6b1fd3818f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "987081c49c244dc085ecade0ccc398ea" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40687 } } peers { permanent_uuid: "25522c36305a47608578bc6b1fd3818f" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 36433 } } peers { permanent_uuid: "716e8cc9acc24153892296893268ae65" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44425 } }
I20260808 23:31:45.105309 6170 raft_consensus.cc:374] T 6deefd2c8e56432cbf3ca5310f3c143e P 25522c36305a47608578bc6b1fd3818f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:45.105461 6170 raft_consensus.cc:729] T 6deefd2c8e56432cbf3ca5310f3c143e P 25522c36305a47608578bc6b1fd3818f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25522c36305a47608578bc6b1fd3818f, State: Initialized, Role: FOLLOWER
I20260808 23:31:45.105607 6169 ts_tablet_manager.cc:1428] T 6deefd2c8e56432cbf3ca5310f3c143e P 987081c49c244dc085ecade0ccc398ea: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260808 23:31:45.105526 6173 raft_consensus.cc:348] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "987081c49c244dc085ecade0ccc398ea" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40687 } } peers { permanent_uuid: "25522c36305a47608578bc6b1fd3818f" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 36433 } } peers { permanent_uuid: "716e8cc9acc24153892296893268ae65" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44425 } }
I20260808 23:31:45.106029 6173 raft_consensus.cc:374] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:45.106173 6173 raft_consensus.cc:729] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 716e8cc9acc24153892296893268ae65, State: Initialized, Role: FOLLOWER
I20260808 23:31:45.106109 6170 consensus_queue.cc:260] T 6deefd2c8e56432cbf3ca5310f3c143e P 25522c36305a47608578bc6b1fd3818f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "987081c49c244dc085ecade0ccc398ea" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40687 } } peers { permanent_uuid: "25522c36305a47608578bc6b1fd3818f" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 36433 } } peers { permanent_uuid: "716e8cc9acc24153892296893268ae65" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44425 } }
I20260808 23:31:45.107211 5962 heartbeater.cc:503] Master 127.9.44.126:33939 was elected leader, sending a full tablet report...
I20260808 23:31:45.107918 6170 ts_tablet_manager.cc:1428] T 6deefd2c8e56432cbf3ca5310f3c143e P 25522c36305a47608578bc6b1fd3818f: Time spent starting tablet: real 0.012s user 0.006s sys 0.005s
I20260808 23:31:45.106820 6173 consensus_queue.cc:260] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "987081c49c244dc085ecade0ccc398ea" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40687 } } peers { permanent_uuid: "25522c36305a47608578bc6b1fd3818f" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 36433 } } peers { permanent_uuid: "716e8cc9acc24153892296893268ae65" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44425 } }
I20260808 23:31:45.109413 6143 heartbeater.cc:503] Master 127.9.44.126:33939 was elected leader, sending a full tablet report...
I20260808 23:31:45.109848 6173 ts_tablet_manager.cc:1428] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
I20260808 23:31:45.191102 6180 raft_consensus.cc:482] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:31:45.191407 6180 raft_consensus.cc:504] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "987081c49c244dc085ecade0ccc398ea" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40687 } } peers { permanent_uuid: "25522c36305a47608578bc6b1fd3818f" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 36433 } } peers { permanent_uuid: "716e8cc9acc24153892296893268ae65" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44425 } }
I20260808 23:31:45.193167 6180 leader_election.cc:288] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 987081c49c244dc085ecade0ccc398ea (127.9.44.65:40687), 25522c36305a47608578bc6b1fd3818f (127.9.44.66:36433)
I20260808 23:31:45.199965 6162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:45.193480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:35291 (local address 127.9.44.65:40687)
0808 23:31:45.193676 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:45.193682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:45.193724 (+ 42us) server_negotiation.cc:407] Connection header received
0808 23:31:45.194021 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:45.194028 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:45.194153 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:45.194251 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:45.194835 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.195784 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.198063 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.199145 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.199269 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:45.199382 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:45.199552 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 23:31:45.199706 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 23:31:45.199973 6146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:45.193353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:40687 (local address 127.9.44.67:35291)
0808 23:31:45.193653 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:45.193673 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:45.193824 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:45.194351 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:45.194358 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:45.194389 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:45.194674 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:45.194687 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.195916 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.195921 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:45.197908 (+ 1987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:45.197925 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.199281 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.199286 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:45.199411 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:45.199437 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 23:31:45.199507 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 23:31:45.199603 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"spinlock_wait_cycles":25984}
I20260808 23:31:45.200495 6164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:45.193832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:39415 (local address 127.9.44.66:36433)
0808 23:31:45.194003 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:45.194009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:45.194442 (+ 433us) server_negotiation.cc:407] Connection header received
0808 23:31:45.194597 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:45.194601 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:45.194671 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:45.194743 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:45.195318 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.196202 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.198466 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.199614 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.199768 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:45.199853 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:45.200167 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 23:31:45.200304 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 23:31:45.201071 6181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:45.193590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:36433 (local address 127.9.44.67:39415)
0808 23:31:45.194341 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:45.194358 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:45.194484 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:45.194778 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:45.194783 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:45.194809 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:45.195145 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:45.195154 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.196360 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.196366 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:45.198294 (+ 1928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:45.198306 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.199798 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.199804 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:45.199972 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:45.200004 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 23:31:45.200674 (+ 670us) client_negotiation.cc:240] Negotiation successful
0808 23:31:45.200805 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":416,"threads_started":1}
I20260808 23:31:45.201032 5715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6deefd2c8e56432cbf3ca5310f3c143e" candidate_uuid: "716e8cc9acc24153892296893268ae65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "987081c49c244dc085ecade0ccc398ea" is_pre_election: true
I20260808 23:31:45.201568 5715 raft_consensus.cc:2454] T 6deefd2c8e56432cbf3ca5310f3c143e P 987081c49c244dc085ecade0ccc398ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 716e8cc9acc24153892296893268ae65 in term 0.
I20260808 23:31:45.202031 5895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6deefd2c8e56432cbf3ca5310f3c143e" candidate_uuid: "716e8cc9acc24153892296893268ae65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25522c36305a47608578bc6b1fd3818f" is_pre_election: true
I20260808 23:31:45.202168 5987 leader_election.cc:302] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 716e8cc9acc24153892296893268ae65, 987081c49c244dc085ecade0ccc398ea; no voters:
I20260808 23:31:45.202503 6180 raft_consensus.cc:2781] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:31:45.202527 5895 raft_consensus.cc:2454] T 6deefd2c8e56432cbf3ca5310f3c143e P 25522c36305a47608578bc6b1fd3818f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 716e8cc9acc24153892296893268ae65 in term 0.
I20260808 23:31:45.202628 6180 raft_consensus.cc:482] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:31:45.202725 6180 raft_consensus.cc:3037] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:45.204943 6180 raft_consensus.cc:504] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "987081c49c244dc085ecade0ccc398ea" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40687 } } peers { permanent_uuid: "25522c36305a47608578bc6b1fd3818f" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 36433 } } peers { permanent_uuid: "716e8cc9acc24153892296893268ae65" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44425 } }
I20260808 23:31:45.205540 6180 leader_election.cc:288] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65 [CANDIDATE]: Term 1 election: Requested vote from peers 987081c49c244dc085ecade0ccc398ea (127.9.44.65:40687), 25522c36305a47608578bc6b1fd3818f (127.9.44.66:36433)
I20260808 23:31:45.205874 5715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6deefd2c8e56432cbf3ca5310f3c143e" candidate_uuid: "716e8cc9acc24153892296893268ae65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "987081c49c244dc085ecade0ccc398ea"
I20260808 23:31:45.205971 5895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6deefd2c8e56432cbf3ca5310f3c143e" candidate_uuid: "716e8cc9acc24153892296893268ae65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25522c36305a47608578bc6b1fd3818f"
I20260808 23:31:45.206039 5715 raft_consensus.cc:3037] T 6deefd2c8e56432cbf3ca5310f3c143e P 987081c49c244dc085ecade0ccc398ea [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:45.206149 5895 raft_consensus.cc:3037] T 6deefd2c8e56432cbf3ca5310f3c143e P 25522c36305a47608578bc6b1fd3818f [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:45.208223 5715 raft_consensus.cc:2454] T 6deefd2c8e56432cbf3ca5310f3c143e P 987081c49c244dc085ecade0ccc398ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 716e8cc9acc24153892296893268ae65 in term 1.
I20260808 23:31:45.208432 5895 raft_consensus.cc:2454] T 6deefd2c8e56432cbf3ca5310f3c143e P 25522c36305a47608578bc6b1fd3818f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 716e8cc9acc24153892296893268ae65 in term 1.
I20260808 23:31:45.208570 5987 leader_election.cc:302] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 716e8cc9acc24153892296893268ae65, 987081c49c244dc085ecade0ccc398ea; no voters:
I20260808 23:31:45.208828 6180 raft_consensus.cc:2781] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:31:45.209233 6180 raft_consensus.cc:686] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65 [term 1 LEADER]: Becoming Leader. State: Replica: 716e8cc9acc24153892296893268ae65, State: Running, Role: LEADER
I20260808 23:31:45.209717 6180 consensus_queue.cc:237] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "987081c49c244dc085ecade0ccc398ea" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40687 } } peers { permanent_uuid: "25522c36305a47608578bc6b1fd3818f" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 36433 } } peers { permanent_uuid: "716e8cc9acc24153892296893268ae65" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44425 } }
I20260808 23:31:45.214349 5508 catalog_manager.cc:5495] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65 reported cstate change: term changed from 0 to 1, leader changed from <none> to 716e8cc9acc24153892296893268ae65 (127.9.44.67). New cstate: current_term: 1 leader_uuid: "716e8cc9acc24153892296893268ae65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "987081c49c244dc085ecade0ccc398ea" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25522c36305a47608578bc6b1fd3818f" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 36433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "716e8cc9acc24153892296893268ae65" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44425 } health_report { overall_health: HEALTHY } } }
W20260808 23:31:45.223052 6145 tablet.cc:2369] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:31:45.253233 5963 tablet.cc:2369] T 6deefd2c8e56432cbf3ca5310f3c143e P 25522c36305a47608578bc6b1fd3818f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:31:45.260480 6167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:45.255586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60594 (local address 127.9.44.67:44425)
0808 23:31:45.255762 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:45.255767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:45.255887 (+ 120us) server_negotiation.cc:407] Connection header received
0808 23:31:45.256046 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:45.256050 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:45.256139 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:45.256265 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:31:45.256564 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.257275 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.258567 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.258972 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.259062 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:45.259500 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:31:45.260026 (+ 526us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:31:45.260150 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:45.260251 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 23:31:45.260286 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 23:31:45.268239 5895 raft_consensus.cc:1261] T 6deefd2c8e56432cbf3ca5310f3c143e P 25522c36305a47608578bc6b1fd3818f [term 1 FOLLOWER]: Refusing update from remote peer 716e8cc9acc24153892296893268ae65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:31:45.268244 5715 raft_consensus.cc:1261] T 6deefd2c8e56432cbf3ca5310f3c143e P 987081c49c244dc085ecade0ccc398ea [term 1 FOLLOWER]: Refusing update from remote peer 716e8cc9acc24153892296893268ae65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:31:45.269249 6180 consensus_queue.cc:1035] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "987081c49c244dc085ecade0ccc398ea" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 40687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:31:45.269542 6182 consensus_queue.cc:1035] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25522c36305a47608578bc6b1fd3818f" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 36433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:31:45.297916 5510 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43864:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:31:45.298179 5510 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43864:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:31:45.301574 5510 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9cc36b10a1004b3a8d6d684508b0fb6e: Sending DeleteTablet for 3 replicas of tablet 6deefd2c8e56432cbf3ca5310f3c143e
I20260808 23:31:45.302563 5687 tablet_service.cc:1514] Processing DeleteTablet for tablet 6deefd2c8e56432cbf3ca5310f3c143e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:31:45 UTC) from {username='slave'} at 127.0.0.1:48152
I20260808 23:31:45.302764 6050 tablet_service.cc:1514] Processing DeleteTablet for tablet 6deefd2c8e56432cbf3ca5310f3c143e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:31:45 UTC) from {username='slave'} at 127.0.0.1:60578
I20260808 23:31:45.302873 5873 tablet_service.cc:1514] Processing DeleteTablet for tablet 6deefd2c8e56432cbf3ca5310f3c143e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:31:45 UTC) from {username='slave'} at 127.0.0.1:43302
I20260808 23:31:45.304651 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 5593
I20260808 23:31:45.315068 6199 tablet_replica.cc:330] T 6deefd2c8e56432cbf3ca5310f3c143e P 25522c36305a47608578bc6b1fd3818f: stopping tablet replica
I20260808 23:31:45.315483 6199 raft_consensus.cc:2229] T 6deefd2c8e56432cbf3ca5310f3c143e P 25522c36305a47608578bc6b1fd3818f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:31:45.315989 6199 raft_consensus.cc:2258] T 6deefd2c8e56432cbf3ca5310f3c143e P 25522c36305a47608578bc6b1fd3818f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:31:45.319433 6199 ts_tablet_manager.cc:1905] T 6deefd2c8e56432cbf3ca5310f3c143e P 25522c36305a47608578bc6b1fd3818f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:31:45.321209 6197 tablet_replica.cc:330] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65: stopping tablet replica
I20260808 23:31:45.321696 6197 raft_consensus.cc:2229] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:31:45.322453 6197 raft_consensus.cc:2258] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:31:45.323937 6196 tablet_replica.cc:330] T 6deefd2c8e56432cbf3ca5310f3c143e P 987081c49c244dc085ecade0ccc398ea: stopping tablet replica
I20260808 23:31:45.324360 6196 raft_consensus.cc:2229] T 6deefd2c8e56432cbf3ca5310f3c143e P 987081c49c244dc085ecade0ccc398ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:31:45.324915 6196 raft_consensus.cc:2258] T 6deefd2c8e56432cbf3ca5310f3c143e P 987081c49c244dc085ecade0ccc398ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:31:45.326431 6199 ts_tablet_manager.cc:1918] T 6deefd2c8e56432cbf3ca5310f3c143e P 25522c36305a47608578bc6b1fd3818f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:31:45.326661 6199 log.cc:1196] T 6deefd2c8e56432cbf3ca5310f3c143e P 25522c36305a47608578bc6b1fd3818f: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/6deefd2c8e56432cbf3ca5310f3c143e
I20260808 23:31:45.326974 6197 ts_tablet_manager.cc:1905] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:31:45.327328 6199 ts_tablet_manager.cc:1939] T 6deefd2c8e56432cbf3ca5310f3c143e P 25522c36305a47608578bc6b1fd3818f: Deleting consensus metadata
I20260808 23:31:45.329123 5492 catalog_manager.cc:4841] TS 25522c36305a47608578bc6b1fd3818f (127.9.44.66:36433): tablet 6deefd2c8e56432cbf3ca5310f3c143e (table test-table [id=d199979c3efb476ba83b30cc4a61defc]) successfully deleted
I20260808 23:31:45.330662 6196 ts_tablet_manager.cc:1905] T 6deefd2c8e56432cbf3ca5310f3c143e P 987081c49c244dc085ecade0ccc398ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:31:45.331434 6162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:45.306422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48154 (local address 127.9.44.65:40687)
0808 23:31:45.306605 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:45.306613 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:45.323246 (+ 16633us) server_negotiation.cc:407] Connection header received
0808 23:31:45.323436 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:45.323441 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:45.323547 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:45.323696 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:45.324353 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.325569 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.327365 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.327966 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.328062 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:45.329831 (+ 1769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:45.329841 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:45.329856 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:45.330548 (+ 692us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:45.330605 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:45.330634 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:45.330724 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:45.330976 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 23:31:45.331080 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 23:31:45.332731 5777 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:45.334528 6197 ts_tablet_manager.cc:1918] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:31:45.334699 6197 log.cc:1196] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/6deefd2c8e56432cbf3ca5310f3c143e
I20260808 23:31:45.335266 6197 ts_tablet_manager.cc:1939] T 6deefd2c8e56432cbf3ca5310f3c143e P 716e8cc9acc24153892296893268ae65: Deleting consensus metadata
I20260808 23:31:45.336813 5492 catalog_manager.cc:4841] TS 716e8cc9acc24153892296893268ae65 (127.9.44.67:44425): tablet 6deefd2c8e56432cbf3ca5310f3c143e (table test-table [id=d199979c3efb476ba83b30cc4a61defc]) successfully deleted
I20260808 23:31:45.444350 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 5593
W20260808 23:31:45.466068 5493 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.44.65:40687 (error 108)
I20260808 23:31:45.466809 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 5787
I20260808 23:31:45.468168 6166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:45.467607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:40687 (local address 127.0.0.1:48158)
0808 23:31:45.467831 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:45.467986 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:40687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260808 23:31:45.468504 5493 catalog_manager.cc:4584] TS 987081c49c244dc085ecade0ccc398ea (127.9.44.65:40687): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6deefd2c8e56432cbf3ca5310f3c143e: Network error: Client connection negotiation failed: client connection to 127.9.44.65:40687: connect: Connection refused (error 111)
I20260808 23:31:45.473989 6164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:45.467612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43314 (local address 127.9.44.66:36433)
0808 23:31:45.467832 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:45.467838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:45.467966 (+ 128us) server_negotiation.cc:407] Connection header received
0808 23:31:45.468146 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:45.468152 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:45.468251 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:45.468346 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:45.468965 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.469705 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.470976 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.471389 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.471507 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:45.472654 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:45.472664 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:45.472678 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:45.473246 (+ 568us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:45.473296 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:45.473317 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:45.473418 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:45.473539 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 23:31:45.473676 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 23:31:45.474578 5956 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:45.522027 6166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:45.521580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:40687 (local address 127.0.0.1:48168)
0808 23:31:45.521769 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:45.521862 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:40687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 23:31:45.523077 6166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:45.522752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:40687 (local address 127.0.0.1:48172)
0808 23:31:45.522901 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:45.522969 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:40687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 23:31:45.583349 6166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:45.582921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:40687 (local address 127.0.0.1:48182)
0808 23:31:45.583110 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:45.583192 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:40687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 23:31:45.584405 6166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:45.584045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:40687 (local address 127.0.0.1:48186)
0808 23:31:45.584197 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:45.584269 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:40687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 23:31:45.621675 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 5787
I20260808 23:31:45.640523 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 5967
I20260808 23:31:45.648610 6167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:45.641385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60596 (local address 127.9.44.67:44425)
0808 23:31:45.641548 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:45.641553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:45.641570 (+ 17us) server_negotiation.cc:407] Connection header received
0808 23:31:45.641783 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:45.641787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:45.641867 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:45.641948 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:45.642479 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.643240 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.645125 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:45.645556 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:45.645656 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:45.647053 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:45.647062 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:45.647080 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:45.647831 (+ 751us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:45.647882 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:45.647904 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:45.647989 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:45.648158 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 23:31:45.648259 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 23:31:45.649333 6132 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:45.665076 6166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:45.664632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:40687 (local address 127.0.0.1:48188)
0808 23:31:45.664828 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:45.664923 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:40687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 23:31:45.666085 6166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:45.665712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:40687 (local address 127.0.0.1:48204)
0808 23:31:45.665900 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:45.665983 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:40687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260808 23:31:45.768671 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 5967
I20260808 23:31:45.796293 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 5475
I20260808 23:31:45.797398 5554 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:45.905051 6166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:45.904365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:40687 (local address 127.0.0.1:48208)
0808 23:31:45.904587 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:45.904692 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:40687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 23:31:45.905342 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 5475
2026-08-08T23:31:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2868 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:31:46 dist-test-slave-6bbx krb5kdc[6215](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:31:46 dist-test-slave-6bbx krb5kdc[6215](info): set up 2 sockets
Aug 08 23:31:46 dist-test-slave-6bbx krb5kdc[6215](info): commencing operation
krb5kdc: starting...
W20260808 23:31:47.117592 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:31:47 dist-test-slave-6bbx krb5kdc[6215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:31:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:31:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:31:47.376976 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:46003
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:44779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:46003
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:31:47.698482 6230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:47.698752 6230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:47.698885 6230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:47.705880 6230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:31:47.705981 6230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:47.706054 6230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:47.706131 6230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:31:47.706187 6230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:31:47.715381 6230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:44779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:46003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:46003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:47.716543 6230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:47.717743 6230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:47.728740 6237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:47.728662 6238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:47.728706 6242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:47.732959 6230 server_base.cc:1034] running on GCE node
I20260808 23:31:47.733683 6230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:47.734952 6230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:47.736158 6230 hybrid_clock.cc:648] HybridClock initialized: now 1786231907736099 us; error 35 us; skew 500 ppm
I20260808 23:31:47.736491 6230 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:31:47 dist-test-slave-6bbx krb5kdc[6215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231907, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:31:47.742588 6230 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:31:47.744073 6230 webserver.cc:460] Webserver started at http://127.9.44.126:36015/ using document root <none> and password file <none>
I20260808 23:31:47.744731 6230 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:47.744846 6230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:47.745119 6230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:47.747983 6230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "075ad88ed8624545baf26133cfba6748"
format_stamp: "Formatted at 2026-08-08 23:31:47 on dist-test-slave-6bbx"
server_key: "3b67b8b80a29c2bc1c773f9d70e5380f"
server_key_iv: "e4bd4a1e4370516749d391e2211b6a6a"
server_key_version: "encryptionkey@0"
I20260808 23:31:47.748685 6230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "075ad88ed8624545baf26133cfba6748"
format_stamp: "Formatted at 2026-08-08 23:31:47 on dist-test-slave-6bbx"
server_key: "3b67b8b80a29c2bc1c773f9d70e5380f"
server_key_iv: "e4bd4a1e4370516749d391e2211b6a6a"
server_key_version: "encryptionkey@0"
I20260808 23:31:47.753414 6230 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 23:31:47.756536 6250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:47.757675 6230 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 23:31:47.757830 6230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "075ad88ed8624545baf26133cfba6748"
format_stamp: "Formatted at 2026-08-08 23:31:47 on dist-test-slave-6bbx"
server_key: "3b67b8b80a29c2bc1c773f9d70e5380f"
server_key_iv: "e4bd4a1e4370516749d391e2211b6a6a"
server_key_version: "encryptionkey@0"
I20260808 23:31:47.757995 6230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:47.781077 6230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:47.784564 6230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:47.784807 6230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:47.801822 6230 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:46003
I20260808 23:31:47.801859 6321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:46003 every 8 connection(s)
I20260808 23:31:47.803021 6230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:31:47.806456 6323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:47.810168 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 6230
I20260808 23:31:47.810374 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:31:47.810904 9393 external_mini_cluster.cc:1428] Setting key 114d92922003e896365d15b75acf1225
I20260808 23:31:47.818835 6323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 075ad88ed8624545baf26133cfba6748: Bootstrap starting.
I20260808 23:31:47.822472 6323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 075ad88ed8624545baf26133cfba6748: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:47.823796 6323 log.cc:824] T 00000000000000000000000000000000 P 075ad88ed8624545baf26133cfba6748: Log is configured to *not* fsync() on all Append() calls
Aug 08 23:31:47 dist-test-slave-6bbx krb5kdc[6215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:31:47.826596 6323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 075ad88ed8624545baf26133cfba6748: No bootstrap required, opened a new log
I20260808 23:31:47.834199 6331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:47.812628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39248 (local address 127.9.44.126:46003)
0808 23:31:47.813392 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:47.813403 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:47.813442 (+ 39us) server_negotiation.cc:407] Connection header received
0808 23:31:47.814979 (+ 1537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:47.815000 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:47.815360 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:47.815658 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:47.817433 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:47.818664 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:47.820455 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:47.821079 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:47.821195 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:47.825967 (+ 4772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:47.825978 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:47.825994 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:47.826036 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:47.829954 (+ 3918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:47.830650 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:47.830658 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:47.830666 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:47.830892 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:47.831208 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:47.831213 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:47.831217 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:47.831919 (+ 702us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:47.832230 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:47.832513 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 23:31:47.832943 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":362,"threads_started":1}
I20260808 23:31:47.836710 6323 raft_consensus.cc:348] T 00000000000000000000000000000000 P 075ad88ed8624545baf26133cfba6748 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075ad88ed8624545baf26133cfba6748" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 46003 } }
I20260808 23:31:47.837131 6323 raft_consensus.cc:374] T 00000000000000000000000000000000 P 075ad88ed8624545baf26133cfba6748 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:47.837239 6323 raft_consensus.cc:729] T 00000000000000000000000000000000 P 075ad88ed8624545baf26133cfba6748 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 075ad88ed8624545baf26133cfba6748, State: Initialized, Role: FOLLOWER
I20260808 23:31:47.837787 6323 consensus_queue.cc:260] T 00000000000000000000000000000000 P 075ad88ed8624545baf26133cfba6748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075ad88ed8624545baf26133cfba6748" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 46003 } }
I20260808 23:31:47.838001 6323 raft_consensus.cc:388] T 00000000000000000000000000000000 P 075ad88ed8624545baf26133cfba6748 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:31:47.838125 6323 raft_consensus.cc:482] T 00000000000000000000000000000000 P 075ad88ed8624545baf26133cfba6748 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:31:47.838284 6323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 075ad88ed8624545baf26133cfba6748 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:47.840340 6323 raft_consensus.cc:504] T 00000000000000000000000000000000 P 075ad88ed8624545baf26133cfba6748 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075ad88ed8624545baf26133cfba6748" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 46003 } }
I20260808 23:31:47.840811 6323 leader_election.cc:302] T 00000000000000000000000000000000 P 075ad88ed8624545baf26133cfba6748 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 075ad88ed8624545baf26133cfba6748; no voters:
I20260808 23:31:47.841354 6323 leader_election.cc:288] T 00000000000000000000000000000000 P 075ad88ed8624545baf26133cfba6748 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:31:47.841472 6333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 075ad88ed8624545baf26133cfba6748 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:31:47.842164 6333 raft_consensus.cc:686] T 00000000000000000000000000000000 P 075ad88ed8624545baf26133cfba6748 [term 1 LEADER]: Becoming Leader. State: Replica: 075ad88ed8624545baf26133cfba6748, State: Running, Role: LEADER
I20260808 23:31:47.842772 6333 consensus_queue.cc:237] T 00000000000000000000000000000000 P 075ad88ed8624545baf26133cfba6748 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075ad88ed8624545baf26133cfba6748" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 46003 } }
I20260808 23:31:47.843321 6323 sys_catalog.cc:564] T 00000000000000000000000000000000 P 075ad88ed8624545baf26133cfba6748 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:31:47.845741 6335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 075ad88ed8624545baf26133cfba6748 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "075ad88ed8624545baf26133cfba6748" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075ad88ed8624545baf26133cfba6748" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 46003 } } }
I20260808 23:31:47.845695 6337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 075ad88ed8624545baf26133cfba6748 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 075ad88ed8624545baf26133cfba6748. Latest consensus state: current_term: 1 leader_uuid: "075ad88ed8624545baf26133cfba6748" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075ad88ed8624545baf26133cfba6748" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 46003 } } }
I20260808 23:31:47.846133 6337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 075ad88ed8624545baf26133cfba6748 [sys.catalog]: This master's current role is: LEADER
I20260808 23:31:47.846143 6335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 075ad88ed8624545baf26133cfba6748 [sys.catalog]: This master's current role is: LEADER
I20260808 23:31:47.847582 6347 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:31:47.852556 6347 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:31:47.861258 6347 catalog_manager.cc:1347] Generated new cluster ID: 1739a0a885474d49993e6cf86c3dc655
I20260808 23:31:47.861370 6347 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:31:47.902436 6347 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:31:47.903725 6347 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:31:47.917773 6347 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 075ad88ed8624545baf26133cfba6748: Generated new TSK 0
I20260808 23:31:47.918560 6347 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.44.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
Entry for principal kudu/127.9.44.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab.
I20260808 23:31:47.988976 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:0
--local_ip_for_outbound_sockets=127.9.44.65
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:46003
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:44779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:31:48.311174 6365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:48.311460 6365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:48.311601 6365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:48.318579 6365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:48.318694 6365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:48.318846 6365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:31:48.328339 6365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:44779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.65.keytab
--principal=kudu/127.9.44.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:46003
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:48.329514 6365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:48.330753 6365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:48.343370 6372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:48.343549 6370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:48.343413 6374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:48.344445 6365 server_base.cc:1034] running on GCE node
I20260808 23:31:48.345067 6365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:48.345908 6365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:48.347116 6365 hybrid_clock.cc:648] HybridClock initialized: now 1786231908347069 us; error 32 us; skew 500 ppm
I20260808 23:31:48.347438 6365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:31:48 dist-test-slave-6bbx krb5kdc[6215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231908, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:31:48.350423 6365 init.cc:377] Logged in from keytab as kudu/127.9.44.65@KRBTEST.COM (short username kudu)
I20260808 23:31:48.351758 6365 webserver.cc:460] Webserver started at http://127.9.44.65:33223/ using document root <none> and password file <none>
I20260808 23:31:48.352401 6365 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:48.352519 6365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:48.352808 6365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:48.355583 6365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-0/data/instance:
uuid: "d93dbb4c030f4c7fb5d61125145f8d18"
format_stamp: "Formatted at 2026-08-08 23:31:48 on dist-test-slave-6bbx"
server_key: "e8fe7f374f2953a9ab4bc1da85101d63"
server_key_iv: "2f4ca29a3e57b28710abcedaa47a5129"
server_key_version: "encryptionkey@0"
I20260808 23:31:48.356398 6365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance:
uuid: "d93dbb4c030f4c7fb5d61125145f8d18"
format_stamp: "Formatted at 2026-08-08 23:31:48 on dist-test-slave-6bbx"
server_key: "e8fe7f374f2953a9ab4bc1da85101d63"
server_key_iv: "2f4ca29a3e57b28710abcedaa47a5129"
server_key_version: "encryptionkey@0"
I20260808 23:31:48.361155 6365 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 23:31:48.364355 6383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:48.365530 6365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:31:48.365687 6365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "d93dbb4c030f4c7fb5d61125145f8d18"
format_stamp: "Formatted at 2026-08-08 23:31:48 on dist-test-slave-6bbx"
server_key: "e8fe7f374f2953a9ab4bc1da85101d63"
server_key_iv: "2f4ca29a3e57b28710abcedaa47a5129"
server_key_version: "encryptionkey@0"
I20260808 23:31:48.365857 6365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:48.377841 6365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:48.381271 6365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:48.381517 6365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:48.382377 6365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:31:48.383561 6365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:31:48.383646 6365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:48.383781 6365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:31:48.383841 6365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:48.413681 6365 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:42507
I20260808 23:31:48.413717 6530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:42507 every 8 connection(s)
I20260808 23:31:48.414870 6365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:31:48.424723 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 6365
I20260808 23:31:48.424913 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance
I20260808 23:31:48.425422 9393 external_mini_cluster.cc:1428] Setting key c2d4551d650379838161ebf0af3a3749
Aug 08 23:31:48 dist-test-slave-6bbx krb5kdc[6215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231908, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.65@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:31:48.434267 6534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:48.417295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:60193 (local address 127.9.44.126:46003)
0808 23:31:48.417807 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:48.417814 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:48.418663 (+ 849us) server_negotiation.cc:407] Connection header received
0808 23:31:48.420525 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:48.420530 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:48.420596 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:48.420683 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:48.422112 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:48.422841 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:48.424326 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:48.424929 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:48.425063 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:48.428377 (+ 3314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:48.428383 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:48.428387 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:48.428424 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:48.431420 (+ 2996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:48.432151 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:48.432157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:48.432160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:48.432304 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:48.432683 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:48.432687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:48.432691 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:48.433007 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:48.433150 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:48.433742 (+ 592us) server_negotiation.cc:299] Negotiation successful
0808 23:31:48.433957 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":279,"threads_started":1}
I20260808 23:31:48.434938 6535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:48.417785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:46003 (local address 127.9.44.65:60193)
0808 23:31:48.418504 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:48.418542 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:48.420267 (+ 1725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:48.420870 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:48.420878 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:48.421360 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:31:48.421952 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:48.421969 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:48.422961 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:48.422965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:48.424216 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:48.424229 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:48.425126 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:48.425132 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:48.425353 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:48.426000 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:31:48.426021 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:31:48.428161 (+ 2140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:31:48.431556 (+ 3395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:31:48.431562 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:31:48.431573 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:31:48.432019 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:31:48.432398 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:31:48.432401 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:31:48.432403 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:31:48.432579 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:31:48.433106 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:31:48.433111 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:31:48.433393 (+ 282us) client_negotiation.cc:769] Sending connection context
0808 23:31:48.433592 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 23:31:48.433937 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":272,"threads_started":1}
I20260808 23:31:48.436399 6532 heartbeater.cc:347] Connected to a master server at 127.9.44.126:46003
I20260808 23:31:48.436705 6532 heartbeater.cc:464] Registering TS with master...
I20260808 23:31:48.437507 6532 heartbeater.cc:511] Master 127.9.44.126:46003 requested a full tablet report, sending...
I20260808 23:31:48.439563 6275 ts_manager.cc:194] Registered new tserver with Master: d93dbb4c030f4c7fb5d61125145f8d18 (127.9.44.65:42507)
I20260808 23:31:48.440991 6275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.65@KRBTEST.COM'} at 127.9.44.65:60193
WARNING: no policy specified for kudu/127.9.44.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
Entry for principal kudu/127.9.44.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab.
I20260808 23:31:48.487610 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:0
--local_ip_for_outbound_sockets=127.9.44.66
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:46003
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:44779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:31:48.814133 6539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:48.814440 6539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:48.814657 6539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:48.825109 6539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:48.825209 6539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:48.825366 6539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:31:48.834610 6539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:44779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.66.keytab
--principal=kudu/127.9.44.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:46003
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:48.835778 6539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:48.837152 6539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:48.850160 6546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:48.850193 6548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:48.852694 6545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:48.853195 6539 server_base.cc:1034] running on GCE node
I20260808 23:31:48.853924 6539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:48.855062 6539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:48.856294 6539 hybrid_clock.cc:648] HybridClock initialized: now 1786231908856243 us; error 35 us; skew 500 ppm
I20260808 23:31:48.856664 6539 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:31:48 dist-test-slave-6bbx krb5kdc[6215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231908, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:31:48.860440 6539 init.cc:377] Logged in from keytab as kudu/127.9.44.66@KRBTEST.COM (short username kudu)
I20260808 23:31:48.862066 6539 webserver.cc:460] Webserver started at http://127.9.44.66:34709/ using document root <none> and password file <none>
I20260808 23:31:48.862726 6539 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:48.862836 6539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:48.863132 6539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:48.865921 6539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-1/data/instance:
uuid: "7bdd0bb86a79494daaa996e263fe8974"
format_stamp: "Formatted at 2026-08-08 23:31:48 on dist-test-slave-6bbx"
server_key: "1a9091858ab5d31afc22c2dc7c85a14b"
server_key_iv: "3d45e84d6455867bcb414b966e86d53c"
server_key_version: "encryptionkey@0"
I20260808 23:31:48.866662 6539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance:
uuid: "7bdd0bb86a79494daaa996e263fe8974"
format_stamp: "Formatted at 2026-08-08 23:31:48 on dist-test-slave-6bbx"
server_key: "1a9091858ab5d31afc22c2dc7c85a14b"
server_key_iv: "3d45e84d6455867bcb414b966e86d53c"
server_key_version: "encryptionkey@0"
I20260808 23:31:48.871869 6539 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260808 23:31:48.875183 6561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:48.876441 6539 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:31:48.876573 6539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "7bdd0bb86a79494daaa996e263fe8974"
format_stamp: "Formatted at 2026-08-08 23:31:48 on dist-test-slave-6bbx"
server_key: "1a9091858ab5d31afc22c2dc7c85a14b"
server_key_iv: "3d45e84d6455867bcb414b966e86d53c"
server_key_version: "encryptionkey@0"
I20260808 23:31:48.876760 6539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:48.896822 6539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:48.900712 6539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:48.900956 6539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:48.901916 6539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:31:48.903198 6539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:31:48.903271 6539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:48.903378 6539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:31:48.903425 6539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:48.947657 6539 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:46511
I20260808 23:31:48.947770 6713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:46511 every 8 connection(s)
I20260808 23:31:48.948961 6539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:31:48.952376 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 6539
I20260808 23:31:48.952594 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance
I20260808 23:31:48.953198 9393 external_mini_cluster.cc:1428] Setting key 30babbafa09ff930d608e8f656af8b61
Aug 08 23:31:48 dist-test-slave-6bbx krb5kdc[6215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231908, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.66@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:31:48.968286 6717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:48.951334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:39941 (local address 127.9.44.126:46003)
0808 23:31:48.951825 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:48.951832 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:48.952867 (+ 1035us) server_negotiation.cc:407] Connection header received
0808 23:31:48.955156 (+ 2289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:48.955161 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:48.955242 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:48.955337 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:48.956821 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:48.957541 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:48.959050 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:48.959473 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:48.959582 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:48.962937 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:48.962942 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:48.962945 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:48.962977 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:48.965401 (+ 2424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:48.966060 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:48.966064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:48.966066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:48.966198 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:48.966601 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:48.966604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:48.966606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:48.966867 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:48.967000 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:48.967657 (+ 657us) server_negotiation.cc:299] Negotiation successful
0808 23:31:48.967902 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":235,"threads_started":1}
I20260808 23:31:48.969159 6718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:48.951877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:46003 (local address 127.9.44.66:39941)
0808 23:31:48.952698 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:48.952751 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:48.954883 (+ 2132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:48.955568 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:48.955576 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:48.956096 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:31:48.956653 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:48.956677 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:48.957688 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:48.957692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:48.958912 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:48.958925 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:48.959709 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:48.959718 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:48.959945 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:48.960562 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:31:48.960585 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:31:48.962735 (+ 2150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:31:48.965528 (+ 2793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:31:48.965536 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:31:48.965551 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:31:48.965938 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:31:48.966301 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:31:48.966306 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:31:48.966310 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:31:48.966486 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:31:48.966990 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:31:48.966996 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:31:48.967302 (+ 306us) client_negotiation.cc:769] Sending connection context
0808 23:31:48.967522 (+ 220us) client_negotiation.cc:240] Negotiation successful
0808 23:31:48.968034 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":291,"threads_started":1}
I20260808 23:31:48.970553 6715 heartbeater.cc:347] Connected to a master server at 127.9.44.126:46003
I20260808 23:31:48.970831 6715 heartbeater.cc:464] Registering TS with master...
I20260808 23:31:48.971495 6715 heartbeater.cc:511] Master 127.9.44.126:46003 requested a full tablet report, sending...
I20260808 23:31:48.973002 6275 ts_manager.cc:194] Registered new tserver with Master: 7bdd0bb86a79494daaa996e263fe8974 (127.9.44.66:46511)
I20260808 23:31:48.974064 6275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.66@KRBTEST.COM'} at 127.9.44.66:39941
WARNING: no policy specified for kudu/127.9.44.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
Entry for principal kudu/127.9.44.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab.
I20260808 23:31:49.015477 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:0
--local_ip_for_outbound_sockets=127.9.44.67
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:46003
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:44779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:31:49.346813 6722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:49.347062 6722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:49.347165 6722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:49.354187 6722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:49.354283 6722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:49.354398 6722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:31:49.363778 6722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:44779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.67.keytab
--principal=kudu/127.9.44.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:46003
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:49.364847 6722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:49.366067 6722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:49.378254 6732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:49.378311 6733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:49.379061 6738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:49.379251 6722 server_base.cc:1034] running on GCE node
I20260808 23:31:49.379936 6722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:49.380851 6722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:49.382050 6722 hybrid_clock.cc:648] HybridClock initialized: now 1786231909381976 us; error 61 us; skew 500 ppm
I20260808 23:31:49.382367 6722 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:31:49 dist-test-slave-6bbx krb5kdc[6215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231909, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:31:49.385342 6722 init.cc:377] Logged in from keytab as kudu/127.9.44.67@KRBTEST.COM (short username kudu)
I20260808 23:31:49.386631 6722 webserver.cc:460] Webserver started at http://127.9.44.67:39075/ using document root <none> and password file <none>
I20260808 23:31:49.387241 6722 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:49.387341 6722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:49.387589 6722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:49.390352 6722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-2/data/instance:
uuid: "8d2b97deb69b4c928846be3ed31d3731"
format_stamp: "Formatted at 2026-08-08 23:31:49 on dist-test-slave-6bbx"
server_key: "cd50ee7c7f619a9b7744a048ecb01ca9"
server_key_iv: "0a900b5f997cef6b3ad8e36801afa101"
server_key_version: "encryptionkey@0"
I20260808 23:31:49.391036 6722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance:
uuid: "8d2b97deb69b4c928846be3ed31d3731"
format_stamp: "Formatted at 2026-08-08 23:31:49 on dist-test-slave-6bbx"
server_key: "cd50ee7c7f619a9b7744a048ecb01ca9"
server_key_iv: "0a900b5f997cef6b3ad8e36801afa101"
server_key_version: "encryptionkey@0"
I20260808 23:31:49.395875 6722 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 23:31:49.399158 6751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:49.400447 6722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:31:49.400583 6722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "8d2b97deb69b4c928846be3ed31d3731"
format_stamp: "Formatted at 2026-08-08 23:31:49 on dist-test-slave-6bbx"
server_key: "cd50ee7c7f619a9b7744a048ecb01ca9"
server_key_iv: "0a900b5f997cef6b3ad8e36801afa101"
server_key_version: "encryptionkey@0"
I20260808 23:31:49.400760 6722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:49.413728 6722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:49.417711 6722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:49.417964 6722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:49.418859 6722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:31:49.420184 6722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:31:49.420271 6722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:49.420390 6722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:31:49.420449 6722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:49.443724 6532 heartbeater.cc:503] Master 127.9.44.126:46003 was elected leader, sending a full tablet report...
I20260808 23:31:49.451175 6722 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:35347
I20260808 23:31:49.451225 6896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:35347 every 8 connection(s)
I20260808 23:31:49.452365 6722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:31:49.459434 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 6722
I20260808 23:31:49.459622 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance
I20260808 23:31:49.460143 9393 external_mini_cluster.cc:1428] Setting key e77ac456554bb0b15d6e8a62c69a3683
Aug 08 23:31:49 dist-test-slave-6bbx krb5kdc[6215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231909, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.67@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:31:49.470943 6717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:49.454566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:45115 (local address 127.9.44.126:46003)
0808 23:31:49.454777 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:49.454783 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:49.455913 (+ 1130us) server_negotiation.cc:407] Connection header received
0808 23:31:49.457631 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:49.457635 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:49.457699 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:49.457788 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:49.459128 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.460001 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.462098 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.462782 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.462875 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:49.465996 (+ 3121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:49.466000 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:49.466004 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:49.466034 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:49.468374 (+ 2340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:49.468989 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:49.468993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:49.468995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:49.469110 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:49.469461 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:49.469465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:49.469467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:49.469828 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:49.469943 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:49.470549 (+ 606us) server_negotiation.cc:299] Negotiation successful
0808 23:31:49.470709 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:31:49.471758 6900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:49.455028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:46003 (local address 127.9.44.67:45115)
0808 23:31:49.455757 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:49.455807 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:49.457406 (+ 1599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:49.457976 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:49.457984 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:49.458451 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:31:49.458994 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:49.459012 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.460154 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.460160 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:49.461958 (+ 1798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:49.461973 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.462987 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.462997 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:49.463200 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:49.463777 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:31:49.463799 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:31:49.465799 (+ 2000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:31:49.468491 (+ 2692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:31:49.468498 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:31:49.468508 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:31:49.468894 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:31:49.469188 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:31:49.469191 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:31:49.469193 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:31:49.469375 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:31:49.469954 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:31:49.469960 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:31:49.470236 (+ 276us) client_negotiation.cc:769] Sending connection context
0808 23:31:49.470453 (+ 217us) client_negotiation.cc:240] Negotiation successful
0808 23:31:49.470784 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":281,"threads_started":1}
I20260808 23:31:49.472899 6897 heartbeater.cc:347] Connected to a master server at 127.9.44.126:46003
I20260808 23:31:49.473150 6897 heartbeater.cc:464] Registering TS with master...
I20260808 23:31:49.473924 6897 heartbeater.cc:511] Master 127.9.44.126:46003 requested a full tablet report, sending...
I20260808 23:31:49.475348 6275 ts_manager.cc:194] Registered new tserver with Master: 8d2b97deb69b4c928846be3ed31d3731 (127.9.44.67:35347)
I20260808 23:31:49.476150 6275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.44.67@KRBTEST.COM'} at 127.9.44.67:45115
I20260808 23:31:49.486805 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:31:49.502806 6717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:49.490871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39258 (local address 127.9.44.126:46003)
0808 23:31:49.491009 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:49.491015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:49.491441 (+ 426us) server_negotiation.cc:407] Connection header received
0808 23:31:49.491599 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:49.491603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:49.491712 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:49.491811 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:49.493159 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.493872 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.495213 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.495611 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.495750 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:49.497932 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:49.497937 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:49.497940 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:49.497969 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:49.500277 (+ 2308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:49.500914 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:49.500918 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:49.500920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:49.501044 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:49.501399 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:49.501404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:49.501406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:49.501709 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:49.501951 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:49.502306 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 23:31:49.502511 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 23:31:49.506203 6275 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39258:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:31:49.512797 6275 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:31:49.546103 6922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:49.524484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58278 (local address 127.9.44.66:46511)
0808 23:31:49.536062 (+ 11578us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:49.536072 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:49.536909 (+ 837us) server_negotiation.cc:407] Connection header received
0808 23:31:49.537019 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:49.537028 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:49.537297 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:49.538035 (+ 738us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:49.538226 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.539228 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.542343 (+ 3115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.544911 (+ 2568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.545037 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:49.545241 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:49.545581 (+ 340us) server_negotiation.cc:299] Negotiation successful
0808 23:31:49.545790 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11414,"thread_start_us":11307,"threads_started":1}
I20260808 23:31:49.546093 6919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:49.524212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:46511 (local address 127.0.0.1:58278)
0808 23:31:49.536475 (+ 12263us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:49.536515 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:49.536696 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:49.537534 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:49.537541 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:49.537565 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:49.537861 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:49.537870 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.539359 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.539364 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:49.542162 (+ 2798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:49.542178 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.545122 (+ 2944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.545129 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:49.545330 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:49.545383 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 23:31:49.545517 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 23:31:49.545724 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12044,"thread_start_us":11896,"threads_started":1}
I20260808 23:31:49.547557 6921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:49.524221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:35347 (local address 127.0.0.1:42510)
0808 23:31:49.539267 (+ 15046us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:49.539284 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:49.539420 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:49.540522 (+ 1102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:49.540527 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:49.540552 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:49.540985 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:49.540999 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.542899 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.542904 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:49.544975 (+ 2071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:49.544990 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.546802 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.546808 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:49.546988 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:49.547020 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 23:31:49.547109 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 23:31:49.547226 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14938,"thread_start_us":14850,"threads_started":1}
I20260808 23:31:49.548213 6920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:49.524280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42510 (local address 127.9.44.67:35347)
0808 23:31:49.539046 (+ 14766us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:49.539054 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:49.539546 (+ 492us) server_negotiation.cc:407] Connection header received
0808 23:31:49.539649 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:49.539656 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:49.539917 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:49.540531 (+ 614us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:49.541198 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.542743 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.545147 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.546603 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.546728 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:49.546939 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:49.547747 (+ 808us) server_negotiation.cc:299] Negotiation successful
0808 23:31:49.547921 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14598,"thread_start_us":14447,"threads_started":1}
I20260808 23:31:49.549952 6630 tablet_service.cc:1467] Processing CreateTablet for tablet 1defbcd434234db2a7467b3706c5bd08 (DEFAULT_TABLE table=test-table [id=71f64a6d14ee4ad280c152294d41e8ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:31:49.551506 6630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1defbcd434234db2a7467b3706c5bd08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:49.552033 6923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:49.539714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60186 (local address 127.9.44.65:42507)
0808 23:31:49.544191 (+ 4477us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:49.544198 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:49.544649 (+ 451us) server_negotiation.cc:407] Connection header received
0808 23:31:49.544831 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:49.544837 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:49.544994 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:49.545169 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:49.545946 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.548173 (+ 2227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.550019 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.551026 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.551135 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:49.551317 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:49.551544 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 23:31:49.551727 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4329,"thread_start_us":4212,"threads_started":1}
I20260808 23:31:49.552034 6812 tablet_service.cc:1467] Processing CreateTablet for tablet 1defbcd434234db2a7467b3706c5bd08 (DEFAULT_TABLE table=test-table [id=71f64a6d14ee4ad280c152294d41e8ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:31:49.552642 6924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:49.540929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:42507 (local address 127.0.0.1:60186)
0808 23:31:49.544536 (+ 3607us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:49.544551 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:49.544667 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:49.545329 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:49.545333 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:49.545359 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:49.545753 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:49.545766 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.548303 (+ 2537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.548309 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:49.549874 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:49.549886 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.551219 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.551226 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:49.551388 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:49.551410 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 23:31:49.551487 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 23:31:49.552312 (+ 825us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3492,"thread_start_us":3333,"threads_started":1}
I20260808 23:31:49.553453 6812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1defbcd434234db2a7467b3706c5bd08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:49.555197 6445 tablet_service.cc:1467] Processing CreateTablet for tablet 1defbcd434234db2a7467b3706c5bd08 (DEFAULT_TABLE table=test-table [id=71f64a6d14ee4ad280c152294d41e8ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:31:49.556213 6445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1defbcd434234db2a7467b3706c5bd08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:49.566325 6925 tablet_bootstrap.cc:492] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974: Bootstrap starting.
I20260808 23:31:49.568207 6928 tablet_bootstrap.cc:492] T 1defbcd434234db2a7467b3706c5bd08 P 8d2b97deb69b4c928846be3ed31d3731: Bootstrap starting.
I20260808 23:31:49.568969 6929 tablet_bootstrap.cc:492] T 1defbcd434234db2a7467b3706c5bd08 P d93dbb4c030f4c7fb5d61125145f8d18: Bootstrap starting.
I20260808 23:31:49.570252 6925 tablet_bootstrap.cc:654] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:49.571504 6925 log.cc:824] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:49.571823 6929 tablet_bootstrap.cc:654] T 1defbcd434234db2a7467b3706c5bd08 P d93dbb4c030f4c7fb5d61125145f8d18: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:49.572501 6928 tablet_bootstrap.cc:654] T 1defbcd434234db2a7467b3706c5bd08 P 8d2b97deb69b4c928846be3ed31d3731: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:49.573012 6929 log.cc:824] T 1defbcd434234db2a7467b3706c5bd08 P d93dbb4c030f4c7fb5d61125145f8d18: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:49.573765 6928 log.cc:824] T 1defbcd434234db2a7467b3706c5bd08 P 8d2b97deb69b4c928846be3ed31d3731: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:49.574898 6925 tablet_bootstrap.cc:492] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974: No bootstrap required, opened a new log
I20260808 23:31:49.575156 6925 ts_tablet_manager.cc:1397] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260808 23:31:49.576445 6929 tablet_bootstrap.cc:492] T 1defbcd434234db2a7467b3706c5bd08 P d93dbb4c030f4c7fb5d61125145f8d18: No bootstrap required, opened a new log
I20260808 23:31:49.576670 6929 ts_tablet_manager.cc:1397] T 1defbcd434234db2a7467b3706c5bd08 P d93dbb4c030f4c7fb5d61125145f8d18: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260808 23:31:49.576856 6928 tablet_bootstrap.cc:492] T 1defbcd434234db2a7467b3706c5bd08 P 8d2b97deb69b4c928846be3ed31d3731: No bootstrap required, opened a new log
I20260808 23:31:49.577099 6928 ts_tablet_manager.cc:1397] T 1defbcd434234db2a7467b3706c5bd08 P 8d2b97deb69b4c928846be3ed31d3731: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260808 23:31:49.583508 6929 raft_consensus.cc:348] T 1defbcd434234db2a7467b3706c5bd08 P d93dbb4c030f4c7fb5d61125145f8d18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2b97deb69b4c928846be3ed31d3731" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 35347 } } peers { permanent_uuid: "7bdd0bb86a79494daaa996e263fe8974" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 46511 } } peers { permanent_uuid: "d93dbb4c030f4c7fb5d61125145f8d18" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42507 } }
I20260808 23:31:49.583928 6929 raft_consensus.cc:374] T 1defbcd434234db2a7467b3706c5bd08 P d93dbb4c030f4c7fb5d61125145f8d18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:49.584036 6929 raft_consensus.cc:729] T 1defbcd434234db2a7467b3706c5bd08 P d93dbb4c030f4c7fb5d61125145f8d18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d93dbb4c030f4c7fb5d61125145f8d18, State: Initialized, Role: FOLLOWER
I20260808 23:31:49.584745 6929 consensus_queue.cc:260] T 1defbcd434234db2a7467b3706c5bd08 P d93dbb4c030f4c7fb5d61125145f8d18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2b97deb69b4c928846be3ed31d3731" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 35347 } } peers { permanent_uuid: "7bdd0bb86a79494daaa996e263fe8974" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 46511 } } peers { permanent_uuid: "d93dbb4c030f4c7fb5d61125145f8d18" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42507 } }
I20260808 23:31:49.585384 6925 raft_consensus.cc:348] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2b97deb69b4c928846be3ed31d3731" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 35347 } } peers { permanent_uuid: "7bdd0bb86a79494daaa996e263fe8974" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 46511 } } peers { permanent_uuid: "d93dbb4c030f4c7fb5d61125145f8d18" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42507 } }
I20260808 23:31:49.585794 6925 raft_consensus.cc:374] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:49.585886 6925 raft_consensus.cc:729] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bdd0bb86a79494daaa996e263fe8974, State: Initialized, Role: FOLLOWER
I20260808 23:31:49.586441 6929 ts_tablet_manager.cc:1428] T 1defbcd434234db2a7467b3706c5bd08 P d93dbb4c030f4c7fb5d61125145f8d18: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260808 23:31:49.586432 6925 consensus_queue.cc:260] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2b97deb69b4c928846be3ed31d3731" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 35347 } } peers { permanent_uuid: "7bdd0bb86a79494daaa996e263fe8974" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 46511 } } peers { permanent_uuid: "d93dbb4c030f4c7fb5d61125145f8d18" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42507 } }
I20260808 23:31:49.587181 6928 raft_consensus.cc:348] T 1defbcd434234db2a7467b3706c5bd08 P 8d2b97deb69b4c928846be3ed31d3731 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2b97deb69b4c928846be3ed31d3731" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 35347 } } peers { permanent_uuid: "7bdd0bb86a79494daaa996e263fe8974" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 46511 } } peers { permanent_uuid: "d93dbb4c030f4c7fb5d61125145f8d18" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42507 } }
I20260808 23:31:49.587466 6715 heartbeater.cc:503] Master 127.9.44.126:46003 was elected leader, sending a full tablet report...
I20260808 23:31:49.587598 6928 raft_consensus.cc:374] T 1defbcd434234db2a7467b3706c5bd08 P 8d2b97deb69b4c928846be3ed31d3731 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:49.587739 6928 raft_consensus.cc:729] T 1defbcd434234db2a7467b3706c5bd08 P 8d2b97deb69b4c928846be3ed31d3731 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d2b97deb69b4c928846be3ed31d3731, State: Initialized, Role: FOLLOWER
I20260808 23:31:49.588191 6925 ts_tablet_manager.cc:1428] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260808 23:31:49.588486 6928 consensus_queue.cc:260] T 1defbcd434234db2a7467b3706c5bd08 P 8d2b97deb69b4c928846be3ed31d3731 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2b97deb69b4c928846be3ed31d3731" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 35347 } } peers { permanent_uuid: "7bdd0bb86a79494daaa996e263fe8974" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 46511 } } peers { permanent_uuid: "d93dbb4c030f4c7fb5d61125145f8d18" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42507 } }
I20260808 23:31:49.590813 6897 heartbeater.cc:503] Master 127.9.44.126:46003 was elected leader, sending a full tablet report...
I20260808 23:31:49.591369 6928 ts_tablet_manager.cc:1428] T 1defbcd434234db2a7467b3706c5bd08 P 8d2b97deb69b4c928846be3ed31d3731: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
W20260808 23:31:49.667069 6533 tablet.cc:2369] T 1defbcd434234db2a7467b3706c5bd08 P d93dbb4c030f4c7fb5d61125145f8d18: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:31:49.700806 6716 tablet.cc:2369] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:31:49.703874 6898 tablet.cc:2369] T 1defbcd434234db2a7467b3706c5bd08 P 8d2b97deb69b4c928846be3ed31d3731: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:31:49.732880 6936 raft_consensus.cc:482] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:31:49.733162 6936 raft_consensus.cc:504] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2b97deb69b4c928846be3ed31d3731" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 35347 } } peers { permanent_uuid: "7bdd0bb86a79494daaa996e263fe8974" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 46511 } } peers { permanent_uuid: "d93dbb4c030f4c7fb5d61125145f8d18" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42507 } }
I20260808 23:31:49.734858 6936 leader_election.cc:288] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d2b97deb69b4c928846be3ed31d3731 (127.9.44.67:35347), d93dbb4c030f4c7fb5d61125145f8d18 (127.9.44.65:42507)
I20260808 23:31:49.741587 6920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:49.735182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:33117 (local address 127.9.44.67:35347)
0808 23:31:49.735403 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:49.735410 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:49.735998 (+ 588us) server_negotiation.cc:407] Connection header received
0808 23:31:49.736194 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:49.736200 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:49.736282 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:49.736384 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:49.737069 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.737987 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.739898 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.740759 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.740875 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:49.740978 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:49.741289 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 23:31:49.741404 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 23:31:49.741727 6938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:49.735171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:35347 (local address 127.9.44.66:33117)
0808 23:31:49.735890 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:49.735910 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:49.736058 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:49.736498 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:49.736503 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:49.736539 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:49.736928 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:49.736941 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.738117 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.738122 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:49.739753 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:49.739767 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.740977 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.740983 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:49.741120 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:49.741145 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 23:31:49.741249 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 23:31:49.741380 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":378,"threads_started":1}
I20260808 23:31:49.742007 6923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:49.736288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:51761 (local address 127.9.44.65:42507)
0808 23:31:49.736474 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:49.736480 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:49.736684 (+ 204us) server_negotiation.cc:407] Connection header received
0808 23:31:49.736832 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:49.736838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:49.736915 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:49.737010 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:49.737558 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.738330 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.740134 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.741101 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.741192 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:49.741304 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:49.741660 (+ 356us) server_negotiation.cc:299] Negotiation successful
0808 23:31:49.741779 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 23:31:49.742328 6939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:49.735963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:42507 (local address 127.9.44.66:51761)
0808 23:31:49.736592 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:49.736605 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:49.736712 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:49.737048 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:49.737052 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:49.737075 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:49.737423 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:49.737436 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.738468 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.738471 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:49.739987 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:49.740002 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.741300 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.741306 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:49.741491 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:49.741529 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 23:31:49.741618 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 23:31:49.742034 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":308,"threads_started":1}
I20260808 23:31:49.742686 6837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1defbcd434234db2a7467b3706c5bd08" candidate_uuid: "7bdd0bb86a79494daaa996e263fe8974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d2b97deb69b4c928846be3ed31d3731" is_pre_election: true
I20260808 23:31:49.743244 6837 raft_consensus.cc:2454] T 1defbcd434234db2a7467b3706c5bd08 P 8d2b97deb69b4c928846be3ed31d3731 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bdd0bb86a79494daaa996e263fe8974 in term 0.
I20260808 23:31:49.743297 6476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1defbcd434234db2a7467b3706c5bd08" candidate_uuid: "7bdd0bb86a79494daaa996e263fe8974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d93dbb4c030f4c7fb5d61125145f8d18" is_pre_election: true
I20260808 23:31:49.743841 6476 raft_consensus.cc:2454] T 1defbcd434234db2a7467b3706c5bd08 P d93dbb4c030f4c7fb5d61125145f8d18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bdd0bb86a79494daaa996e263fe8974 in term 0.
I20260808 23:31:49.743942 6565 leader_election.cc:302] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bdd0bb86a79494daaa996e263fe8974, 8d2b97deb69b4c928846be3ed31d3731; no voters:
I20260808 23:31:49.744325 6936 raft_consensus.cc:2781] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:31:49.744478 6936 raft_consensus.cc:482] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:31:49.744591 6936 raft_consensus.cc:3037] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:49.746678 6936 raft_consensus.cc:504] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2b97deb69b4c928846be3ed31d3731" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 35347 } } peers { permanent_uuid: "7bdd0bb86a79494daaa996e263fe8974" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 46511 } } peers { permanent_uuid: "d93dbb4c030f4c7fb5d61125145f8d18" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42507 } }
I20260808 23:31:49.747346 6936 leader_election.cc:288] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974 [CANDIDATE]: Term 1 election: Requested vote from peers 8d2b97deb69b4c928846be3ed31d3731 (127.9.44.67:35347), d93dbb4c030f4c7fb5d61125145f8d18 (127.9.44.65:42507)
I20260808 23:31:49.747809 6476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1defbcd434234db2a7467b3706c5bd08" candidate_uuid: "7bdd0bb86a79494daaa996e263fe8974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d93dbb4c030f4c7fb5d61125145f8d18"
I20260808 23:31:49.747812 6837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1defbcd434234db2a7467b3706c5bd08" candidate_uuid: "7bdd0bb86a79494daaa996e263fe8974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d2b97deb69b4c928846be3ed31d3731"
I20260808 23:31:49.748001 6476 raft_consensus.cc:3037] T 1defbcd434234db2a7467b3706c5bd08 P d93dbb4c030f4c7fb5d61125145f8d18 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:49.748018 6837 raft_consensus.cc:3037] T 1defbcd434234db2a7467b3706c5bd08 P 8d2b97deb69b4c928846be3ed31d3731 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:49.751048 6837 raft_consensus.cc:2454] T 1defbcd434234db2a7467b3706c5bd08 P 8d2b97deb69b4c928846be3ed31d3731 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bdd0bb86a79494daaa996e263fe8974 in term 1.
I20260808 23:31:49.751048 6476 raft_consensus.cc:2454] T 1defbcd434234db2a7467b3706c5bd08 P d93dbb4c030f4c7fb5d61125145f8d18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bdd0bb86a79494daaa996e263fe8974 in term 1.
I20260808 23:31:49.751430 6565 leader_election.cc:302] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bdd0bb86a79494daaa996e263fe8974, 8d2b97deb69b4c928846be3ed31d3731; no voters:
I20260808 23:31:49.751690 6936 raft_consensus.cc:2781] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:31:49.752099 6936 raft_consensus.cc:686] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974 [term 1 LEADER]: Becoming Leader. State: Replica: 7bdd0bb86a79494daaa996e263fe8974, State: Running, Role: LEADER
I20260808 23:31:49.752537 6936 consensus_queue.cc:237] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2b97deb69b4c928846be3ed31d3731" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 35347 } } peers { permanent_uuid: "7bdd0bb86a79494daaa996e263fe8974" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 46511 } } peers { permanent_uuid: "d93dbb4c030f4c7fb5d61125145f8d18" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42507 } }
I20260808 23:31:49.756608 6275 catalog_manager.cc:5495] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bdd0bb86a79494daaa996e263fe8974 (127.9.44.66). New cstate: current_term: 1 leader_uuid: "7bdd0bb86a79494daaa996e263fe8974" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2b97deb69b4c928846be3ed31d3731" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 35347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bdd0bb86a79494daaa996e263fe8974" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 46511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d93dbb4c030f4c7fb5d61125145f8d18" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42507 } health_report { overall_health: UNKNOWN } } }
I20260808 23:31:49.777642 6922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:49.772597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58288 (local address 127.9.44.66:46511)
0808 23:31:49.772782 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:49.772788 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:49.772875 (+ 87us) server_negotiation.cc:407] Connection header received
0808 23:31:49.773069 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:49.773073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:49.773152 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:49.773273 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:31:49.773651 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.774338 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.775684 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.776086 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.776191 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:49.776644 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:31:49.777094 (+ 450us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:31:49.777220 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:49.777330 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 23:31:49.777404 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 23:31:49.785418 6476 raft_consensus.cc:1261] T 1defbcd434234db2a7467b3706c5bd08 P d93dbb4c030f4c7fb5d61125145f8d18 [term 1 FOLLOWER]: Refusing update from remote peer 7bdd0bb86a79494daaa996e263fe8974: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:31:49.785418 6837 raft_consensus.cc:1261] T 1defbcd434234db2a7467b3706c5bd08 P 8d2b97deb69b4c928846be3ed31d3731 [term 1 FOLLOWER]: Refusing update from remote peer 7bdd0bb86a79494daaa996e263fe8974: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:31:49.786343 6942 consensus_queue.cc:1035] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d2b97deb69b4c928846be3ed31d3731" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 35347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:31:49.786664 6936 consensus_queue.cc:1035] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d93dbb4c030f4c7fb5d61125145f8d18" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 42507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:31:49.798025 6947 mvcc.cc:205] Tried to move back new op lower bound from 7316405902471593984 to 7316405902349836288. Current Snapshot: MvccSnapshot[applied={T|T < 7316405902471593984}]
I20260808 23:31:49.809664 6274 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:31:49.809996 6274 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:31:49.814333 6274 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 075ad88ed8624545baf26133cfba6748: Sending DeleteTablet for 3 replicas of tablet 1defbcd434234db2a7467b3706c5bd08
I20260808 23:31:49.815507 6445 tablet_service.cc:1514] Processing DeleteTablet for tablet 1defbcd434234db2a7467b3706c5bd08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:31:49 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:60186
I20260808 23:31:49.816053 6630 tablet_service.cc:1514] Processing DeleteTablet for tablet 1defbcd434234db2a7467b3706c5bd08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:31:49 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:58278
I20260808 23:31:49.816186 6812 tablet_service.cc:1514] Processing DeleteTablet for tablet 1defbcd434234db2a7467b3706c5bd08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:31:49 UTC) from {username='kudu', principal='kudu/127.9.44.126@KRBTEST.COM'} at 127.0.0.1:42510
I20260808 23:31:49.817893 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 6365
I20260808 23:31:49.828248 6961 tablet_replica.cc:330] T 1defbcd434234db2a7467b3706c5bd08 P 8d2b97deb69b4c928846be3ed31d3731: stopping tablet replica
I20260808 23:31:49.828696 6961 raft_consensus.cc:2229] T 1defbcd434234db2a7467b3706c5bd08 P 8d2b97deb69b4c928846be3ed31d3731 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:31:49.829219 6961 raft_consensus.cc:2258] T 1defbcd434234db2a7467b3706c5bd08 P 8d2b97deb69b4c928846be3ed31d3731 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:31:49.830595 6958 tablet_replica.cc:330] T 1defbcd434234db2a7467b3706c5bd08 P d93dbb4c030f4c7fb5d61125145f8d18: stopping tablet replica
I20260808 23:31:49.830966 6958 raft_consensus.cc:2229] T 1defbcd434234db2a7467b3706c5bd08 P d93dbb4c030f4c7fb5d61125145f8d18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:31:49.831154 6961 ts_tablet_manager.cc:1905] T 1defbcd434234db2a7467b3706c5bd08 P 8d2b97deb69b4c928846be3ed31d3731: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:31:49.831439 6958 raft_consensus.cc:2258] T 1defbcd434234db2a7467b3706c5bd08 P d93dbb4c030f4c7fb5d61125145f8d18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:31:49.833361 6958 ts_tablet_manager.cc:1905] T 1defbcd434234db2a7467b3706c5bd08 P d93dbb4c030f4c7fb5d61125145f8d18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:31:49.835399 6959 tablet_replica.cc:330] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974: stopping tablet replica
I20260808 23:31:49.835803 6959 raft_consensus.cc:2229] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:31:49.836445 6959 raft_consensus.cc:2258] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:31:49.838122 6961 ts_tablet_manager.cc:1918] T 1defbcd434234db2a7467b3706c5bd08 P 8d2b97deb69b4c928846be3ed31d3731: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:31:49.838303 6961 log.cc:1196] T 1defbcd434234db2a7467b3706c5bd08 P 8d2b97deb69b4c928846be3ed31d3731: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/1defbcd434234db2a7467b3706c5bd08
I20260808 23:31:49.838841 6961 ts_tablet_manager.cc:1939] T 1defbcd434234db2a7467b3706c5bd08 P 8d2b97deb69b4c928846be3ed31d3731: Deleting consensus metadata
I20260808 23:31:49.840188 6959 ts_tablet_manager.cc:1905] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:31:49.840634 6958 ts_tablet_manager.cc:1918] T 1defbcd434234db2a7467b3706c5bd08 P d93dbb4c030f4c7fb5d61125145f8d18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:31:49.840582 6259 catalog_manager.cc:4841] TS 8d2b97deb69b4c928846be3ed31d3731 (127.9.44.67:35347): tablet 1defbcd434234db2a7467b3706c5bd08 (table test-table [id=71f64a6d14ee4ad280c152294d41e8ab]) successfully deleted
I20260808 23:31:49.840787 6958 log.cc:1196] T 1defbcd434234db2a7467b3706c5bd08 P d93dbb4c030f4c7fb5d61125145f8d18: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-0/wal/wals/1defbcd434234db2a7467b3706c5bd08
I20260808 23:31:49.841311 6958 ts_tablet_manager.cc:1939] T 1defbcd434234db2a7467b3706c5bd08 P d93dbb4c030f4c7fb5d61125145f8d18: Deleting consensus metadata
I20260808 23:31:49.842857 6259 catalog_manager.cc:4841] TS d93dbb4c030f4c7fb5d61125145f8d18 (127.9.44.65:42507): tablet 1defbcd434234db2a7467b3706c5bd08 (table test-table [id=71f64a6d14ee4ad280c152294d41e8ab]) successfully deleted
Aug 08 23:31:49 dist-test-slave-6bbx krb5kdc[6215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.65@KRBTEST.COM
I20260808 23:31:49.847561 6959 ts_tablet_manager.cc:1918] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:31:49.847780 6959 log.cc:1196] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/1defbcd434234db2a7467b3706c5bd08
I20260808 23:31:49.848310 6959 ts_tablet_manager.cc:1939] T 1defbcd434234db2a7467b3706c5bd08 P 7bdd0bb86a79494daaa996e263fe8974: Deleting consensus metadata
I20260808 23:31:49.849823 6256 catalog_manager.cc:4841] TS 7bdd0bb86a79494daaa996e263fe8974 (127.9.44.66:46511): tablet 1defbcd434234db2a7467b3706c5bd08 (table test-table [id=71f64a6d14ee4ad280c152294d41e8ab]) successfully deleted
I20260808 23:31:49.851646 6923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:49.818840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60202 (local address 127.9.44.65:42507)
0808 23:31:49.818999 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:49.819005 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:49.830396 (+ 11391us) server_negotiation.cc:407] Connection header received
0808 23:31:49.830637 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:49.830643 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:49.830722 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:49.830834 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:49.832849 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.833986 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.836050 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.836663 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.836764 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:49.844546 (+ 7782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:49.844560 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:49.844576 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:49.844616 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:49.848389 (+ 3773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:49.849245 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:49.849254 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:49.849257 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:49.849424 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:49.849869 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:49.849875 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:49.849879 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:49.850354 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:49.850670 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:49.850873 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 23:31:49.851262 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 23:31:49.852295 6524 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:49.940445 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 6365
I20260808 23:31:49.974151 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 6539
Aug 08 23:31:49 dist-test-slave-6bbx krb5kdc[6215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.66@KRBTEST.COM
I20260808 23:31:49.990549 6922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:49.974911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58292 (local address 127.9.44.66:46511)
0808 23:31:49.975378 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:49.975387 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:49.975401 (+ 14us) server_negotiation.cc:407] Connection header received
0808 23:31:49.975449 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:49.975453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:49.975505 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:49.975595 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:49.976897 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.977702 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.978990 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:49.979430 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:49.979545 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:49.984059 (+ 4514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:49.984076 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:49.984094 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:49.984141 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:49.987912 (+ 3771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:49.988508 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:49.988518 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:49.988522 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:49.988719 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:49.989041 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:49.989047 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:49.989050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:49.989465 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:49.989794 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:49.989943 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 23:31:49.990167 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365}
I20260808 23:31:49.991247 6708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:50.101468 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 6539
I20260808 23:31:50.127022 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 6722
Aug 08 23:31:50 dist-test-slave-6bbx krb5kdc[6215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.67@KRBTEST.COM
I20260808 23:31:50.140904 6920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:50.127797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42518 (local address 127.9.44.67:35347)
0808 23:31:50.127976 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:50.127981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:50.128031 (+ 50us) server_negotiation.cc:407] Connection header received
0808 23:31:50.128258 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:50.128262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:50.128339 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:50.128424 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:50.129655 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:50.130352 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:50.131476 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:50.131964 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:50.132056 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:50.136066 (+ 4010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:50.136079 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:50.136096 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:50.136129 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:50.138518 (+ 2389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:50.139110 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:50.139116 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:50.139119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:50.139258 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:50.139524 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:50.139527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:50.139529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:50.139977 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:50.140199 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:50.140436 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 23:31:50.140586 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 23:31:50.141515 6892 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:50.227689 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 6722
I20260808 23:31:50.252555 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 6230
I20260808 23:31:50.253676 6317 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:50.360890 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 6230
2026-08-08T23:31:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4457 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T23:31:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:31:50Z Disabled control of system clock
I20260808 23:31:50.451464 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:44239
--webserver_interface=127.9.44.126
--webserver_port=0
--builtin_ntp_servers=127.9.44.84:38645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:44239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 23:31:50.772699 6974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:50.772974 6974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:50.773059 6974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:50.779908 6974 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:31:50.779997 6974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:50.780041 6974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:50.780095 6974 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:31:50.780138 6974 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:31:50.788802 6974 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:38645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:44239
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:44239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:50.789887 6974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:50.791535 6974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:50.802603 6980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:50.802457 6983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:50.802541 6981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:50.803565 6974 server_base.cc:1034] running on GCE node
I20260808 23:31:50.804291 6974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:50.805557 6974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:50.806770 6974 hybrid_clock.cc:648] HybridClock initialized: now 1786231910806718 us; error 44 us; skew 500 ppm
I20260808 23:31:50.807066 6974 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:31:50.809142 6974 webserver.cc:460] Webserver started at http://127.9.44.126:43687/ using document root <none> and password file <none>
I20260808 23:31:50.809734 6974 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:50.809818 6974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:50.810066 6974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:50.812772 6974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "35c85c51deb040cebe80ad84f4bc8c82"
format_stamp: "Formatted at 2026-08-08 23:31:50 on dist-test-slave-6bbx"
server_key: "d5f2f1349ad519a5451309e4b226c471"
server_key_iv: "cbb63694d83889e6e1b4d5097db206c9"
server_key_version: "encryptionkey@0"
I20260808 23:31:50.813469 6974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "35c85c51deb040cebe80ad84f4bc8c82"
format_stamp: "Formatted at 2026-08-08 23:31:50 on dist-test-slave-6bbx"
server_key: "d5f2f1349ad519a5451309e4b226c471"
server_key_iv: "cbb63694d83889e6e1b4d5097db206c9"
server_key_version: "encryptionkey@0"
I20260808 23:31:50.818066 6974 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 23:31:50.821305 6996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:50.822536 6974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 23:31:50.822706 6974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "35c85c51deb040cebe80ad84f4bc8c82"
format_stamp: "Formatted at 2026-08-08 23:31:50 on dist-test-slave-6bbx"
server_key: "d5f2f1349ad519a5451309e4b226c471"
server_key_iv: "cbb63694d83889e6e1b4d5097db206c9"
server_key_version: "encryptionkey@0"
I20260808 23:31:50.822894 6974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:50.864204 6974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:50.864980 6974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:50.865259 6974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:50.882176 6974 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:44239
I20260808 23:31:50.882170 7075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:44239 every 8 connection(s)
I20260808 23:31:50.883450 6974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:31:50.887142 7076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:50.893481 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 6974
I20260808 23:31:50.893679 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:31:50.894245 9393 external_mini_cluster.cc:1428] Setting key ffd8db1eb0ff338f6f3923ce980cee5b
I20260808 23:31:50.898317 7076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35c85c51deb040cebe80ad84f4bc8c82: Bootstrap starting.
I20260808 23:31:50.901444 7076 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35c85c51deb040cebe80ad84f4bc8c82: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:50.902426 7076 log.cc:824] T 00000000000000000000000000000000 P 35c85c51deb040cebe80ad84f4bc8c82: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:50.905138 7076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35c85c51deb040cebe80ad84f4bc8c82: No bootstrap required, opened a new log
I20260808 23:31:50.907398 7081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:50.896230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33350 (local address 127.9.44.126:44239)
0808 23:31:50.897164 (+ 934us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:50.897175 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:50.897222 (+ 47us) server_negotiation.cc:407] Connection header received
0808 23:31:50.899123 (+ 1901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:50.899147 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:50.899574 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:50.899982 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:50.900561 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:50.901865 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:50.903225 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:50.903918 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:50.904096 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:50.905254 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:50.905265 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:50.905284 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:50.905809 (+ 525us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:50.905886 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:50.905905 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:50.906002 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:50.906233 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 23:31:50.906500 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":318,"threads_started":1}
I20260808 23:31:50.912597 7076 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35c85c51deb040cebe80ad84f4bc8c82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c85c51deb040cebe80ad84f4bc8c82" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44239 } }
I20260808 23:31:50.912925 7076 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35c85c51deb040cebe80ad84f4bc8c82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:50.913015 7076 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35c85c51deb040cebe80ad84f4bc8c82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35c85c51deb040cebe80ad84f4bc8c82, State: Initialized, Role: FOLLOWER
I20260808 23:31:50.913511 7076 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35c85c51deb040cebe80ad84f4bc8c82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c85c51deb040cebe80ad84f4bc8c82" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44239 } }
I20260808 23:31:50.913735 7076 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35c85c51deb040cebe80ad84f4bc8c82 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:31:50.913856 7076 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35c85c51deb040cebe80ad84f4bc8c82 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:31:50.914004 7076 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35c85c51deb040cebe80ad84f4bc8c82 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:50.916889 7076 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35c85c51deb040cebe80ad84f4bc8c82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c85c51deb040cebe80ad84f4bc8c82" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44239 } }
I20260808 23:31:50.917454 7076 leader_election.cc:302] T 00000000000000000000000000000000 P 35c85c51deb040cebe80ad84f4bc8c82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35c85c51deb040cebe80ad84f4bc8c82; no voters:
I20260808 23:31:50.918555 7076 leader_election.cc:288] T 00000000000000000000000000000000 P 35c85c51deb040cebe80ad84f4bc8c82 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:31:50.918637 7084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35c85c51deb040cebe80ad84f4bc8c82 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:31:50.919323 7084 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35c85c51deb040cebe80ad84f4bc8c82 [term 1 LEADER]: Becoming Leader. State: Replica: 35c85c51deb040cebe80ad84f4bc8c82, State: Running, Role: LEADER
I20260808 23:31:50.919849 7084 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35c85c51deb040cebe80ad84f4bc8c82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c85c51deb040cebe80ad84f4bc8c82" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44239 } }
I20260808 23:31:50.920429 7076 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35c85c51deb040cebe80ad84f4bc8c82 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:31:50.922421 7088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35c85c51deb040cebe80ad84f4bc8c82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35c85c51deb040cebe80ad84f4bc8c82. Latest consensus state: current_term: 1 leader_uuid: "35c85c51deb040cebe80ad84f4bc8c82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c85c51deb040cebe80ad84f4bc8c82" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44239 } } }
I20260808 23:31:50.922397 7087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35c85c51deb040cebe80ad84f4bc8c82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35c85c51deb040cebe80ad84f4bc8c82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c85c51deb040cebe80ad84f4bc8c82" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44239 } } }
I20260808 23:31:50.922897 7088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35c85c51deb040cebe80ad84f4bc8c82 [sys.catalog]: This master's current role is: LEADER
I20260808 23:31:50.923053 7087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35c85c51deb040cebe80ad84f4bc8c82 [sys.catalog]: This master's current role is: LEADER
I20260808 23:31:50.924897 7092 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:31:50.929844 7092 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:31:50.939073 7092 catalog_manager.cc:1347] Generated new cluster ID: d435c7cd75ce47edb6e8b45f64ca8195
I20260808 23:31:50.939172 7092 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:31:50.959998 7092 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:31:50.961627 7092 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:31:50.970604 7092 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35c85c51deb040cebe80ad84f4bc8c82: Generated new TSK 0
I20260808 23:31:50.971491 7092 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:31:50.975175 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.65:0
--local_ip_for_outbound_sockets=127.9.44.65
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:44239
--builtin_ntp_servers=127.9.44.84:38645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 23:31:51.305006 7109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:51.305261 7109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:51.305380 7109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:51.312383 7109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:51.312487 7109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:51.312623 7109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.65
I20260808 23:31:51.321420 7109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:38645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.44.65
--webserver_port=0
--tserver_master_addrs=127.9.44.126:44239
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.65
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:51.322639 7109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:51.324400 7109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:51.336908 7118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:51.336944 7119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:51.337044 7122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:51.337848 7109 server_base.cc:1034] running on GCE node
I20260808 23:31:51.338406 7109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:51.339273 7109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:51.340467 7109 hybrid_clock.cc:648] HybridClock initialized: now 1786231911340412 us; error 46 us; skew 500 ppm
I20260808 23:31:51.340759 7109 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:31:51.342896 7109 webserver.cc:460] Webserver started at http://127.9.44.65:41063/ using document root <none> and password file <none>
I20260808 23:31:51.343537 7109 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:51.343683 7109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:51.344003 7109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:51.346709 7109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-0/data/instance:
uuid: "cea7762abf5e4c02abc787a91edbff4d"
format_stamp: "Formatted at 2026-08-08 23:31:51 on dist-test-slave-6bbx"
server_key: "ee017c4584fe33b01aaaffc27a6c4842"
server_key_iv: "5dd19aea92da190342b9d3fbea33ab42"
server_key_version: "encryptionkey@0"
I20260808 23:31:51.347479 7109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance:
uuid: "cea7762abf5e4c02abc787a91edbff4d"
format_stamp: "Formatted at 2026-08-08 23:31:51 on dist-test-slave-6bbx"
server_key: "ee017c4584fe33b01aaaffc27a6c4842"
server_key_iv: "5dd19aea92da190342b9d3fbea33ab42"
server_key_version: "encryptionkey@0"
I20260808 23:31:51.352445 7109 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260808 23:31:51.355785 7130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:51.357017 7109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 23:31:51.357169 7109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-0/wal
uuid: "cea7762abf5e4c02abc787a91edbff4d"
format_stamp: "Formatted at 2026-08-08 23:31:51 on dist-test-slave-6bbx"
server_key: "ee017c4584fe33b01aaaffc27a6c4842"
server_key_iv: "5dd19aea92da190342b9d3fbea33ab42"
server_key_version: "encryptionkey@0"
I20260808 23:31:51.357365 7109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:51.391006 7109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:51.391765 7109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:51.392019 7109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:51.392935 7109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:31:51.394330 7109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:31:51.394428 7109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:51.394534 7109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:31:51.394605 7109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:51.424660 7109 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.65:39651
I20260808 23:31:51.424732 7293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.65:39651 every 8 connection(s)
I20260808 23:31:51.426211 7109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-0/data/info.pb
I20260808 23:31:51.435105 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 7109
I20260808 23:31:51.435256 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-0/wal/instance
I20260808 23:31:51.435654 9393 external_mini_cluster.cc:1428] Setting key c42b566faed4199a3080d5e850466268
I20260808 23:31:51.438244 7297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:51.428684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.65:33725 (local address 127.9.44.126:44239)
0808 23:31:51.429094 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:51.429098 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:51.430245 (+ 1147us) server_negotiation.cc:407] Connection header received
0808 23:31:51.432052 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:51.432057 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:51.432145 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:51.432240 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:51.433257 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:51.433984 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:51.435548 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:51.436052 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:51.436180 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:51.437476 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:51.437482 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:51.437485 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:51.437501 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:51.437541 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:51.437553 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:51.437624 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:51.437832 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 23:31:51.437921 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":213,"threads_started":1}
I20260808 23:31:51.438282 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.66:0
--local_ip_for_outbound_sockets=127.9.44.66
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:44239
--builtin_ntp_servers=127.9.44.84:38645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 23:31:51.439113 7299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:51.429264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:44239 (local address 127.9.44.65:33725)
0808 23:31:51.430097 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:51.430131 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:51.431811 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:51.432484 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:51.432493 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:51.432573 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:31:51.433097 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:51.433114 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:51.434123 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:51.434127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:51.435414 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:51.435427 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:51.436285 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:51.436293 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:51.436505 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:51.437174 (+ 669us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:31:51.437178 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:31:51.437236 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:31:51.437314 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:31:51.437638 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:31:51.437649 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:31:51.437658 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 23:31:51.437809 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 23:31:51.438148 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":256,"threads_started":1}
I20260808 23:31:51.440482 7294 heartbeater.cc:347] Connected to a master server at 127.9.44.126:44239
I20260808 23:31:51.440753 7294 heartbeater.cc:464] Registering TS with master...
I20260808 23:31:51.441407 7294 heartbeater.cc:511] Master 127.9.44.126:44239 requested a full tablet report, sending...
I20260808 23:31:51.443454 7019 ts_manager.cc:194] Registered new tserver with Master: cea7762abf5e4c02abc787a91edbff4d (127.9.44.65:39651)
I20260808 23:31:51.445325 7019 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.44.65:33725
W20260808 23:31:51.765748 7302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:51.766000 7302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:51.766083 7302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:51.773034 7302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:51.773128 7302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:51.773236 7302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.66
I20260808 23:31:51.781865 7302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:38645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.44.66
--webserver_port=0
--tserver_master_addrs=127.9.44.126:44239
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.66
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:51.783008 7302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:51.784705 7302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:51.797039 7312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:51.797128 7310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:51.797106 7309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:51.798117 7302 server_base.cc:1034] running on GCE node
I20260808 23:31:51.798697 7302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:51.799556 7302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:51.800766 7302 hybrid_clock.cc:648] HybridClock initialized: now 1786231911800727 us; error 50 us; skew 500 ppm
I20260808 23:31:51.801066 7302 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:31:51.803130 7302 webserver.cc:460] Webserver started at http://127.9.44.66:40443/ using document root <none> and password file <none>
I20260808 23:31:51.803793 7302 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:51.803910 7302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:51.804214 7302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:51.806928 7302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-1/data/instance:
uuid: "4427eccb16214c1fa4fe914eb1b6f285"
format_stamp: "Formatted at 2026-08-08 23:31:51 on dist-test-slave-6bbx"
server_key: "11bd8828e7728a306f82ae3d6e9b86ae"
server_key_iv: "44c8250878260a7d58c0ebf091407cce"
server_key_version: "encryptionkey@0"
I20260808 23:31:51.807699 7302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance:
uuid: "4427eccb16214c1fa4fe914eb1b6f285"
format_stamp: "Formatted at 2026-08-08 23:31:51 on dist-test-slave-6bbx"
server_key: "11bd8828e7728a306f82ae3d6e9b86ae"
server_key_iv: "44c8250878260a7d58c0ebf091407cce"
server_key_version: "encryptionkey@0"
I20260808 23:31:51.812572 7302 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 23:31:51.815744 7319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:51.816915 7302 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 23:31:51.817057 7302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-1/wal
uuid: "4427eccb16214c1fa4fe914eb1b6f285"
format_stamp: "Formatted at 2026-08-08 23:31:51 on dist-test-slave-6bbx"
server_key: "11bd8828e7728a306f82ae3d6e9b86ae"
server_key_iv: "44c8250878260a7d58c0ebf091407cce"
server_key_version: "encryptionkey@0"
I20260808 23:31:51.817246 7302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:51.832836 7302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:51.833549 7302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:51.833796 7302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:51.834656 7302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:31:51.836056 7302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:31:51.836166 7302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:51.836272 7302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:31:51.836341 7302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:51.866736 7302 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.66:38897
I20260808 23:31:51.866808 7480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.66:38897 every 8 connection(s)
I20260808 23:31:51.867992 7302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-1/data/info.pb
I20260808 23:31:51.868971 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 7302
I20260808 23:31:51.869150 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-1/wal/instance
I20260808 23:31:51.869658 9393 external_mini_cluster.cc:1428] Setting key 3b97a202cd58a01a45a8841744b1ac84
I20260808 23:31:51.872378 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.44.67:0
--local_ip_for_outbound_sockets=127.9.44.67
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:44239
--builtin_ntp_servers=127.9.44.84:38645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 23:31:51.880282 7297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:51.870572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:52269 (local address 127.9.44.126:44239)
0808 23:31:51.870773 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:51.870779 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:51.872275 (+ 1496us) server_negotiation.cc:407] Connection header received
0808 23:31:51.874052 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:51.874058 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:51.874138 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:51.874241 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:51.875259 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:51.876190 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:51.877721 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:51.878222 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:51.878345 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:51.879510 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:51.879516 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:51.879520 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:51.879542 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:51.879580 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:51.879595 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:51.879712 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:51.879889 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 23:31:51.879980 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 23:31:51.881127 7484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:51.871089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:44239 (local address 127.9.44.66:52269)
0808 23:31:51.872103 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:51.872158 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:51.873780 (+ 1622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:51.874430 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:51.874438 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:51.874521 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:31:51.875072 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:51.875089 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:51.876342 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:51.876346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:51.877578 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:51.877591 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:51.878398 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:51.878403 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:51.878598 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:51.879227 (+ 629us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:31:51.879229 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:31:51.879267 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:31:51.879336 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:31:51.879708 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:31:51.879716 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:31:51.879721 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 23:31:51.879856 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 23:31:51.880146 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":305,"threads_started":1}
I20260808 23:31:51.882629 7481 heartbeater.cc:347] Connected to a master server at 127.9.44.126:44239
I20260808 23:31:51.882920 7481 heartbeater.cc:464] Registering TS with master...
I20260808 23:31:51.883622 7481 heartbeater.cc:511] Master 127.9.44.126:44239 requested a full tablet report, sending...
I20260808 23:31:51.884994 7019 ts_manager.cc:194] Registered new tserver with Master: 4427eccb16214c1fa4fe914eb1b6f285 (127.9.44.66:38897)
I20260808 23:31:51.885844 7019 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.44.66:52269
W20260808 23:31:52.209363 7485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:52.209609 7485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:52.209710 7485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:52.216617 7485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:52.216709 7485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:52.216815 7485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.44.67
I20260808 23:31:52.225494 7485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:38645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.44.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.44.67
--webserver_port=0
--tserver_master_addrs=127.9.44.126:44239
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.44.67
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:52.226581 7485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:52.228223 7485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:52.240445 7491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:52.240491 7492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:52.240399 7497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:52.241317 7485 server_base.cc:1034] running on GCE node
I20260808 23:31:52.241858 7485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:52.242726 7485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:52.243919 7485 hybrid_clock.cc:648] HybridClock initialized: now 1786231912243868 us; error 41 us; skew 500 ppm
I20260808 23:31:52.244227 7485 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 23:31:52.246316 7485 webserver.cc:460] Webserver started at http://127.9.44.67:34777/ using document root <none> and password file <none>
I20260808 23:31:52.246955 7485 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:52.247058 7485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:52.247367 7485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:52.250119 7485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-2/data/instance:
uuid: "9edd2328d74645289d916f0ead8bce37"
format_stamp: "Formatted at 2026-08-08 23:31:52 on dist-test-slave-6bbx"
server_key: "beeac02b1a24a55be32af55471f5986d"
server_key_iv: "7aa88d5fcebcc94c6c63411c9e8d3127"
server_key_version: "encryptionkey@0"
I20260808 23:31:52.250886 7485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance:
uuid: "9edd2328d74645289d916f0ead8bce37"
format_stamp: "Formatted at 2026-08-08 23:31:52 on dist-test-slave-6bbx"
server_key: "beeac02b1a24a55be32af55471f5986d"
server_key_iv: "7aa88d5fcebcc94c6c63411c9e8d3127"
server_key_version: "encryptionkey@0"
I20260808 23:31:52.255609 7485 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 23:31:52.258838 7504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:52.260113 7485 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:31:52.260272 7485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-2/wal
uuid: "9edd2328d74645289d916f0ead8bce37"
format_stamp: "Formatted at 2026-08-08 23:31:52 on dist-test-slave-6bbx"
server_key: "beeac02b1a24a55be32af55471f5986d"
server_key_iv: "7aa88d5fcebcc94c6c63411c9e8d3127"
server_key_version: "encryptionkey@0"
I20260808 23:31:52.260479 7485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:52.288501 7485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:52.289201 7485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:52.289434 7485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:52.290299 7485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 23:31:52.291644 7485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 23:31:52.291757 7485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:52.291868 7485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 23:31:52.291939 7485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:52.321977 7485 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.67:44719
I20260808 23:31:52.322044 7658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.67:44719 every 8 connection(s)
I20260808 23:31:52.323112 7485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-2/data/info.pb
I20260808 23:31:52.333232 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 7485
I20260808 23:31:52.333400 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-2/wal/instance
I20260808 23:31:52.333913 9393 external_mini_cluster.cc:1428] Setting key 94c0ea01300e8f71c900df7e5bdfb247
I20260808 23:31:52.334911 7297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:52.325441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:52625 (local address 127.9.44.126:44239)
0808 23:31:52.325615 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:52.325620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:52.326989 (+ 1369us) server_negotiation.cc:407] Connection header received
0808 23:31:52.328743 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:52.328748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:52.328816 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:52.328909 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:52.329841 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.330624 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.332209 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.332638 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.332729 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:52.334094 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:52.334100 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:52.334103 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:52.334121 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:52.334158 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:52.334170 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:52.334248 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:52.334459 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 23:31:52.334560 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 23:31:52.335903 7662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:52.325895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:44239 (local address 127.9.44.67:52625)
0808 23:31:52.326818 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:52.326858 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:52.328518 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:52.329082 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:52.329090 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:52.329190 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:31:52.329709 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:52.329727 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.330760 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.330764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:52.332085 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:52.332098 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.332830 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.332837 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:52.333110 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:52.333794 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 23:31:52.333797 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:31:52.333833 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 23:31:52.333899 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:31:52.334261 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:31:52.334272 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:31:52.334281 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 23:31:52.334426 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 23:31:52.334777 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":342,"threads_started":1}
I20260808 23:31:52.337196 7659 heartbeater.cc:347] Connected to a master server at 127.9.44.126:44239
I20260808 23:31:52.337452 7659 heartbeater.cc:464] Registering TS with master...
I20260808 23:31:52.338101 7659 heartbeater.cc:511] Master 127.9.44.126:44239 requested a full tablet report, sending...
I20260808 23:31:52.339370 7016 ts_manager.cc:194] Registered new tserver with Master: 9edd2328d74645289d916f0ead8bce37 (127.9.44.67:44719)
I20260808 23:31:52.340286 7016 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.44.67:52625
I20260808 23:31:52.348729 9393 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 23:31:52.357944 7297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:52.352503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33356 (local address 127.9.44.126:44239)
0808 23:31:52.352675 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:52.352680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:52.353094 (+ 414us) server_negotiation.cc:407] Connection header received
0808 23:31:52.353244 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:52.353249 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:52.353321 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:52.353400 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:52.353900 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.354541 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.355898 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.356300 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.356408 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:52.357311 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:52.357316 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:52.357319 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:52.357334 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:52.357365 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:52.357376 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:52.357446 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:52.357575 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 23:31:52.357660 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 23:31:52.361196 7016 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 23:31:52.368017 7016 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 23:31:52.387583 7677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:52.378568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:44719 (local address 127.0.0.1:48066)
0808 23:31:52.379112 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:52.379145 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:52.379262 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:52.380421 (+ 1159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:52.380428 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:52.380471 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:52.380938 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:52.380956 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.383044 (+ 2088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.383049 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:52.385344 (+ 2295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:52.385357 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.386639 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.386646 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:52.386888 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:52.386941 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 23:31:52.387057 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 23:31:52.387228 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":1,"thread_start_us":243,"threads_started":1}
I20260808 23:31:52.388617 7678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:52.378767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48066 (local address 127.9.44.67:44719)
0808 23:31:52.379809 (+ 1042us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:52.379817 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:52.379855 (+ 38us) server_negotiation.cc:407] Connection header received
0808 23:31:52.379951 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:52.379958 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:52.380182 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:52.381103 (+ 921us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:52.381391 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.382910 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.385536 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.386431 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.386553 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:52.386822 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:52.388105 (+ 1283us) server_negotiation.cc:299] Negotiation successful
0808 23:31:52.388267 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":848,"thread_start_us":1088,"threads_started":1}
I20260808 23:31:52.389217 7679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:52.379594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:39651 (local address 127.0.0.1:55156)
0808 23:31:52.380404 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:52.380430 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:52.380556 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:52.382306 (+ 1750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:52.382310 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:52.382336 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:52.382675 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:52.382685 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.384569 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.384574 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:52.386635 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:52.386652 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.388520 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.388526 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:52.388684 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:52.388717 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 23:31:52.388793 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 23:31:52.388925 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"thread_start_us":1517,"threads_started":1}
I20260808 23:31:52.389394 7681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:52.380195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55156 (local address 127.9.44.65:39651)
0808 23:31:52.381723 (+ 1528us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:52.381740 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:52.381770 (+ 30us) server_negotiation.cc:407] Connection header received
0808 23:31:52.381870 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:52.381877 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:52.382084 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:52.382856 (+ 772us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:52.383075 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.384442 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.386929 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.388322 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.388454 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:52.388682 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:52.388885 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 23:31:52.389050 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1371,"thread_start_us":1223,"threads_started":1}
I20260808 23:31:52.389792 7680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:52.380131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:38897 (local address 127.0.0.1:45378)
0808 23:31:52.381102 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:52.381123 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:52.381232 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:52.382721 (+ 1489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:52.382725 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:52.382748 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:52.383170 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:52.383181 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.385085 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.385090 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:52.387190 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:52.387202 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.389156 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.389162 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:52.389336 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:52.389377 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 23:31:52.389463 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 23:31:52.389577 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":864,"thread_start_us":724,"threads_started":1}
I20260808 23:31:52.390478 7682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:52.380866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45378 (local address 127.9.44.66:38897)
0808 23:31:52.382304 (+ 1438us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:52.382312 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:52.382336 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:31:52.382411 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:52.382416 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:52.382550 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:52.382724 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:52.383401 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.384942 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.387421 (+ 2479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.388699 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.388825 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:52.389028 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:52.390008 (+ 980us) server_negotiation.cc:299] Negotiation successful
0808 23:31:52.390160 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1306,"thread_start_us":1211,"threads_started":1}
I20260808 23:31:52.392405 7568 tablet_service.cc:1467] Processing CreateTablet for tablet ee660a66183d464c95713ae247078be1 (DEFAULT_TABLE table=test-table [id=194d25f3fba541c095f309e4ead2810b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:31:52.393553 7389 tablet_service.cc:1467] Processing CreateTablet for tablet ee660a66183d464c95713ae247078be1 (DEFAULT_TABLE table=test-table [id=194d25f3fba541c095f309e4ead2810b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:31:52.393430 7198 tablet_service.cc:1467] Processing CreateTablet for tablet ee660a66183d464c95713ae247078be1 (DEFAULT_TABLE table=test-table [id=194d25f3fba541c095f309e4ead2810b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 23:31:52.393930 7568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee660a66183d464c95713ae247078be1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:52.394601 7389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee660a66183d464c95713ae247078be1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:52.394748 7198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee660a66183d464c95713ae247078be1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:52.405737 7683 tablet_bootstrap.cc:492] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285: Bootstrap starting.
I20260808 23:31:52.408061 7684 tablet_bootstrap.cc:492] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37: Bootstrap starting.
I20260808 23:31:52.409475 7686 tablet_bootstrap.cc:492] T ee660a66183d464c95713ae247078be1 P cea7762abf5e4c02abc787a91edbff4d: Bootstrap starting.
I20260808 23:31:52.411257 7683 tablet_bootstrap.cc:654] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:52.411688 7684 tablet_bootstrap.cc:654] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:52.412346 7686 tablet_bootstrap.cc:654] T ee660a66183d464c95713ae247078be1 P cea7762abf5e4c02abc787a91edbff4d: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:52.412583 7683 log.cc:824] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:52.412883 7684 log.cc:824] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:52.413376 7686 log.cc:824] T ee660a66183d464c95713ae247078be1 P cea7762abf5e4c02abc787a91edbff4d: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:52.415584 7683 tablet_bootstrap.cc:492] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285: No bootstrap required, opened a new log
I20260808 23:31:52.415736 7684 tablet_bootstrap.cc:492] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37: No bootstrap required, opened a new log
I20260808 23:31:52.415887 7683 ts_tablet_manager.cc:1397] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260808 23:31:52.416026 7684 ts_tablet_manager.cc:1397] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260808 23:31:52.416595 7686 tablet_bootstrap.cc:492] T ee660a66183d464c95713ae247078be1 P cea7762abf5e4c02abc787a91edbff4d: No bootstrap required, opened a new log
I20260808 23:31:52.416844 7686 ts_tablet_manager.cc:1397] T ee660a66183d464c95713ae247078be1 P cea7762abf5e4c02abc787a91edbff4d: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 23:31:52.423602 7683 raft_consensus.cc:348] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edd2328d74645289d916f0ead8bce37" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44719 } } peers { permanent_uuid: "cea7762abf5e4c02abc787a91edbff4d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39651 } } peers { permanent_uuid: "4427eccb16214c1fa4fe914eb1b6f285" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38897 } }
I20260808 23:31:52.424100 7683 raft_consensus.cc:374] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:52.424239 7683 raft_consensus.cc:729] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4427eccb16214c1fa4fe914eb1b6f285, State: Initialized, Role: FOLLOWER
I20260808 23:31:52.424887 7683 consensus_queue.cc:260] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edd2328d74645289d916f0ead8bce37" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44719 } } peers { permanent_uuid: "cea7762abf5e4c02abc787a91edbff4d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39651 } } peers { permanent_uuid: "4427eccb16214c1fa4fe914eb1b6f285" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38897 } }
I20260808 23:31:52.426231 7684 raft_consensus.cc:348] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edd2328d74645289d916f0ead8bce37" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44719 } } peers { permanent_uuid: "cea7762abf5e4c02abc787a91edbff4d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39651 } } peers { permanent_uuid: "4427eccb16214c1fa4fe914eb1b6f285" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38897 } }
I20260808 23:31:52.426658 7684 raft_consensus.cc:374] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:52.426748 7481 heartbeater.cc:503] Master 127.9.44.126:44239 was elected leader, sending a full tablet report...
I20260808 23:31:52.426779 7684 raft_consensus.cc:729] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9edd2328d74645289d916f0ead8bce37, State: Initialized, Role: FOLLOWER
I20260808 23:31:52.427338 7683 ts_tablet_manager.cc:1428] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285: Time spent starting tablet: real 0.011s user 0.010s sys 0.001s
I20260808 23:31:52.427492 7684 consensus_queue.cc:260] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edd2328d74645289d916f0ead8bce37" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44719 } } peers { permanent_uuid: "cea7762abf5e4c02abc787a91edbff4d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39651 } } peers { permanent_uuid: "4427eccb16214c1fa4fe914eb1b6f285" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38897 } }
I20260808 23:31:52.428591 7686 raft_consensus.cc:348] T ee660a66183d464c95713ae247078be1 P cea7762abf5e4c02abc787a91edbff4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edd2328d74645289d916f0ead8bce37" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44719 } } peers { permanent_uuid: "cea7762abf5e4c02abc787a91edbff4d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39651 } } peers { permanent_uuid: "4427eccb16214c1fa4fe914eb1b6f285" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38897 } }
I20260808 23:31:52.429078 7686 raft_consensus.cc:374] T ee660a66183d464c95713ae247078be1 P cea7762abf5e4c02abc787a91edbff4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:52.429328 7686 raft_consensus.cc:729] T ee660a66183d464c95713ae247078be1 P cea7762abf5e4c02abc787a91edbff4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cea7762abf5e4c02abc787a91edbff4d, State: Initialized, Role: FOLLOWER
I20260808 23:31:52.430051 7686 consensus_queue.cc:260] T ee660a66183d464c95713ae247078be1 P cea7762abf5e4c02abc787a91edbff4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edd2328d74645289d916f0ead8bce37" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44719 } } peers { permanent_uuid: "cea7762abf5e4c02abc787a91edbff4d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39651 } } peers { permanent_uuid: "4427eccb16214c1fa4fe914eb1b6f285" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38897 } }
I20260808 23:31:52.440395 7294 heartbeater.cc:503] Master 127.9.44.126:44239 was elected leader, sending a full tablet report...
I20260808 23:31:52.440833 7659 heartbeater.cc:503] Master 127.9.44.126:44239 was elected leader, sending a full tablet report...
I20260808 23:31:52.441092 7686 ts_tablet_manager.cc:1428] T ee660a66183d464c95713ae247078be1 P cea7762abf5e4c02abc787a91edbff4d: Time spent starting tablet: real 0.024s user 0.014s sys 0.008s
I20260808 23:31:52.441877 7684 ts_tablet_manager.cc:1428] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37: Time spent starting tablet: real 0.026s user 0.017s sys 0.008s
W20260808 23:31:52.574709 7661 tablet.cc:2369] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 23:31:52.620323 7482 tablet.cc:2369] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:31:52.653116 7694 raft_consensus.cc:482] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:31:52.653416 7694 raft_consensus.cc:504] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edd2328d74645289d916f0ead8bce37" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44719 } } peers { permanent_uuid: "cea7762abf5e4c02abc787a91edbff4d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39651 } } peers { permanent_uuid: "4427eccb16214c1fa4fe914eb1b6f285" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38897 } }
I20260808 23:31:52.655107 7694 leader_election.cc:288] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cea7762abf5e4c02abc787a91edbff4d (127.9.44.65:39651), 4427eccb16214c1fa4fe914eb1b6f285 (127.9.44.66:38897)
I20260808 23:31:52.661038 7692 raft_consensus.cc:482] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 23:31:52.661389 7692 raft_consensus.cc:504] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edd2328d74645289d916f0ead8bce37" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44719 } } peers { permanent_uuid: "cea7762abf5e4c02abc787a91edbff4d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39651 } } peers { permanent_uuid: "4427eccb16214c1fa4fe914eb1b6f285" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38897 } }
I20260808 23:31:52.661940 7662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:52.655469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.66:38897 (local address 127.9.44.67:51269)
0808 23:31:52.655818 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:52.655834 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:52.655969 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:52.656386 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:52.656391 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:52.656420 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:52.656615 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:52.656626 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.657841 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.657845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:52.659850 (+ 2005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:52.659869 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.661191 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.661197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:52.661368 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:52.661402 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 23:31:52.661500 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 23:31:52.661638 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222}
I20260808 23:31:52.661986 7682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:52.655516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:51269 (local address 127.9.44.66:38897)
0808 23:31:52.655744 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:52.655750 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:52.655911 (+ 161us) server_negotiation.cc:407] Connection header received
0808 23:31:52.656116 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:52.656121 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:52.656225 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:52.656321 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:52.656768 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.657706 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.660004 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.661025 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.661154 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:52.661269 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:52.661553 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 23:31:52.661692 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 23:31:52.662649 7681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:52.655543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.67:41483 (local address 127.9.44.65:39651)
0808 23:31:52.655771 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:52.655778 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:52.656483 (+ 705us) server_negotiation.cc:407] Connection header received
0808 23:31:52.656689 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:52.656694 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:52.656792 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:52.656911 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:52.657384 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.658188 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.660401 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.661550 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.661664 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:52.661744 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:52.662284 (+ 540us) server_negotiation.cc:299] Negotiation successful
0808 23:31:52.662415 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 23:31:52.662698 7696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:52.655591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:39651 (local address 127.9.44.67:41483)
0808 23:31:52.656375 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:52.656397 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:52.656574 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:52.656919 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:52.656922 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:52.656942 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:52.657234 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:52.657241 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.658325 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.658330 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:52.660233 (+ 1903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:52.660244 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.661850 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.661867 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:52.662045 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:52.662078 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 23:31:52.662225 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 23:31:52.662344 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":4,"spinlock_wait_cycles":24448,"thread_start_us":429,"threads_started":1}
I20260808 23:31:52.662961 7419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee660a66183d464c95713ae247078be1" candidate_uuid: "9edd2328d74645289d916f0ead8bce37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4427eccb16214c1fa4fe914eb1b6f285" is_pre_election: true
I20260808 23:31:52.663446 7419 raft_consensus.cc:2454] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9edd2328d74645289d916f0ead8bce37 in term 0.
I20260808 23:31:52.663528 7692 leader_election.cc:288] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9edd2328d74645289d916f0ead8bce37 (127.9.44.67:44719), cea7762abf5e4c02abc787a91edbff4d (127.9.44.65:39651)
I20260808 23:31:52.663743 7232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee660a66183d464c95713ae247078be1" candidate_uuid: "9edd2328d74645289d916f0ead8bce37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea7762abf5e4c02abc787a91edbff4d" is_pre_election: true
I20260808 23:31:52.664342 7232 raft_consensus.cc:2454] T ee660a66183d464c95713ae247078be1 P cea7762abf5e4c02abc787a91edbff4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9edd2328d74645289d916f0ead8bce37 in term 0.
I20260808 23:31:52.664467 7507 leader_election.cc:302] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4427eccb16214c1fa4fe914eb1b6f285, 9edd2328d74645289d916f0ead8bce37; no voters:
I20260808 23:31:52.665128 7694 raft_consensus.cc:2781] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 23:31:52.665355 7694 raft_consensus.cc:482] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 23:31:52.665524 7694 raft_consensus.cc:3037] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:52.668428 7694 raft_consensus.cc:504] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edd2328d74645289d916f0ead8bce37" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44719 } } peers { permanent_uuid: "cea7762abf5e4c02abc787a91edbff4d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39651 } } peers { permanent_uuid: "4427eccb16214c1fa4fe914eb1b6f285" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38897 } }
I20260808 23:31:52.669109 7694 leader_election.cc:288] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37 [CANDIDATE]: Term 1 election: Requested vote from peers cea7762abf5e4c02abc787a91edbff4d (127.9.44.65:39651), 4427eccb16214c1fa4fe914eb1b6f285 (127.9.44.66:38897)
I20260808 23:31:52.669494 7419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee660a66183d464c95713ae247078be1" candidate_uuid: "9edd2328d74645289d916f0ead8bce37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4427eccb16214c1fa4fe914eb1b6f285"
I20260808 23:31:52.669531 7232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee660a66183d464c95713ae247078be1" candidate_uuid: "9edd2328d74645289d916f0ead8bce37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea7762abf5e4c02abc787a91edbff4d"
I20260808 23:31:52.669672 7419 raft_consensus.cc:3037] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:52.669721 7232 raft_consensus.cc:3037] T ee660a66183d464c95713ae247078be1 P cea7762abf5e4c02abc787a91edbff4d [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:52.672678 7419 raft_consensus.cc:2454] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9edd2328d74645289d916f0ead8bce37 in term 1.
I20260808 23:31:52.672894 7681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:52.664832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:40817 (local address 127.9.44.65:39651)
0808 23:31:52.664969 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:52.664974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:52.666106 (+ 1132us) server_negotiation.cc:407] Connection header received
0808 23:31:52.666291 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:52.666296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:52.666362 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:52.666450 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:52.667020 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.668037 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.670537 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.671832 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.671944 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:52.672032 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:52.672518 (+ 486us) server_negotiation.cc:299] Negotiation successful
0808 23:31:52.672634 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 23:31:52.673022 7232 raft_consensus.cc:2454] T ee660a66183d464c95713ae247078be1 P cea7762abf5e4c02abc787a91edbff4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9edd2328d74645289d916f0ead8bce37 in term 1.
I20260808 23:31:52.673303 7507 leader_election.cc:302] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4427eccb16214c1fa4fe914eb1b6f285, 9edd2328d74645289d916f0ead8bce37; no voters:
I20260808 23:31:52.673527 7698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:52.664122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:39651 (local address 127.9.44.66:40817)
0808 23:31:52.665996 (+ 1874us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:52.666013 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:52.666163 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:52.666485 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:52.666490 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:52.666516 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:52.666850 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:52.666861 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.668191 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.668196 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:52.670316 (+ 2120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:52.670332 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.672155 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.672161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:52.672335 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:52.672364 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 23:31:52.673093 (+ 729us) client_negotiation.cc:240] Negotiation successful
0808 23:31:52.673233 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1744,"mutex_wait_us":1,"thread_start_us":1556,"threads_started":1}
I20260808 23:31:52.673547 7678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:52.664483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.44.66:48603 (local address 127.9.44.67:44719)
0808 23:31:52.664664 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:52.664670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:52.665451 (+ 781us) server_negotiation.cc:407] Connection header received
0808 23:31:52.665719 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:52.665724 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:52.665789 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:52.665850 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 23:31:52.666573 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.667479 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.671305 (+ 3826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.672495 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.672586 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:52.672670 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:52.673190 (+ 520us) server_negotiation.cc:299] Negotiation successful
0808 23:31:52.673315 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 23:31:52.673800 7694 raft_consensus.cc:2781] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:31:52.674005 7232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee660a66183d464c95713ae247078be1" candidate_uuid: "4427eccb16214c1fa4fe914eb1b6f285" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cea7762abf5e4c02abc787a91edbff4d" is_pre_election: true
I20260808 23:31:52.674192 7697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:52.664080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.67:44719 (local address 127.9.44.66:48603)
0808 23:31:52.665349 (+ 1269us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:52.665374 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 23:31:52.665561 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:31:52.666015 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:31:52.666018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:31:52.666039 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 23:31:52.666412 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:52.666426 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.667652 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.667657 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:52.671078 (+ 3421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:31:52.671097 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.672832 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.672839 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:31:52.672997 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:52.673029 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 23:31:52.673746 (+ 717us) client_negotiation.cc:240] Negotiation successful
0808 23:31:52.673871 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1082,"mutex_wait_us":25,"thread_start_us":1255,"threads_started":1}
I20260808 23:31:52.674263 7232 raft_consensus.cc:2379] T ee660a66183d464c95713ae247078be1 P cea7762abf5e4c02abc787a91edbff4d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4427eccb16214c1fa4fe914eb1b6f285 in current term 1: Already voted for candidate 9edd2328d74645289d916f0ead8bce37 in this term.
I20260808 23:31:52.674371 7694 raft_consensus.cc:686] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37 [term 1 LEADER]: Becoming Leader. State: Replica: 9edd2328d74645289d916f0ead8bce37, State: Running, Role: LEADER
I20260808 23:31:52.674955 7694 consensus_queue.cc:237] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edd2328d74645289d916f0ead8bce37" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44719 } } peers { permanent_uuid: "cea7762abf5e4c02abc787a91edbff4d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39651 } } peers { permanent_uuid: "4427eccb16214c1fa4fe914eb1b6f285" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38897 } }
I20260808 23:31:52.675242 7599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee660a66183d464c95713ae247078be1" candidate_uuid: "4427eccb16214c1fa4fe914eb1b6f285" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9edd2328d74645289d916f0ead8bce37" is_pre_election: true
I20260808 23:31:52.676258 7324 leader_election.cc:302] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4427eccb16214c1fa4fe914eb1b6f285; no voters: 9edd2328d74645289d916f0ead8bce37, cea7762abf5e4c02abc787a91edbff4d
I20260808 23:31:52.676716 7692 raft_consensus.cc:2726] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 23:31:52.678558 7295 tablet.cc:2369] T ee660a66183d464c95713ae247078be1 P cea7762abf5e4c02abc787a91edbff4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:31:52.679646 7016 catalog_manager.cc:5495] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9edd2328d74645289d916f0ead8bce37 (127.9.44.67). New cstate: current_term: 1 leader_uuid: "9edd2328d74645289d916f0ead8bce37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9edd2328d74645289d916f0ead8bce37" member_type: VOTER last_known_addr { host: "127.9.44.67" port: 44719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cea7762abf5e4c02abc787a91edbff4d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4427eccb16214c1fa4fe914eb1b6f285" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38897 } health_report { overall_health: UNKNOWN } } }
I20260808 23:31:52.762014 7678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:52.755620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48076 (local address 127.9.44.67:44719)
0808 23:31:52.755909 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:52.755916 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:52.755933 (+ 17us) server_negotiation.cc:407] Connection header received
0808 23:31:52.756038 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:52.756043 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:52.756195 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:52.756361 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:31:52.756781 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.757894 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.759387 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.759981 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.760110 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:52.760742 (+ 632us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:31:52.761427 (+ 685us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:31:52.761608 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:52.761665 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 23:31:52.761727 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156}
I20260808 23:31:52.771092 7232 raft_consensus.cc:1261] T ee660a66183d464c95713ae247078be1 P cea7762abf5e4c02abc787a91edbff4d [term 1 FOLLOWER]: Refusing update from remote peer 9edd2328d74645289d916f0ead8bce37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:31:52.771111 7419 raft_consensus.cc:1261] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285 [term 1 FOLLOWER]: Refusing update from remote peer 9edd2328d74645289d916f0ead8bce37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:31:52.772087 7700 consensus_queue.cc:1035] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cea7762abf5e4c02abc787a91edbff4d" member_type: VOTER last_known_addr { host: "127.9.44.65" port: 39651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:31:52.772486 7694 consensus_queue.cc:1035] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4427eccb16214c1fa4fe914eb1b6f285" member_type: VOTER last_known_addr { host: "127.9.44.66" port: 38897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:31:52.782693 7707 mvcc.cc:205] Tried to move back new op lower bound from 7316405914699395072 to 7316405914321039360. Current Snapshot: MvccSnapshot[applied={T|T < 7316405914699395072}]
I20260808 23:31:52.788736 7704 mvcc.cc:205] Tried to move back new op lower bound from 7316405914699395072 to 7316405914321039360. Current Snapshot: MvccSnapshot[applied={T|T < 7316405914699395072 or (T in {7316405914699395072})}]
I20260808 23:31:52.789045 7706 mvcc.cc:205] Tried to move back new op lower bound from 7316405914699395072 to 7316405914321039360. Current Snapshot: MvccSnapshot[applied={T|T < 7316405914699395072}]
I20260808 23:31:52.795168 7016 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:31:52.795373 7016 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 23:31:52.798559 7016 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 35c85c51deb040cebe80ad84f4bc8c82: Sending DeleteTablet for 3 replicas of tablet ee660a66183d464c95713ae247078be1
I20260808 23:31:52.799593 7198 tablet_service.cc:1514] Processing DeleteTablet for tablet ee660a66183d464c95713ae247078be1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:31:52 UTC) from {username='slave'} at 127.0.0.1:55156
I20260808 23:31:52.799958 7568 tablet_service.cc:1514] Processing DeleteTablet for tablet ee660a66183d464c95713ae247078be1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:31:52 UTC) from {username='slave'} at 127.0.0.1:48066
I20260808 23:31:52.800215 7389 tablet_service.cc:1514] Processing DeleteTablet for tablet ee660a66183d464c95713ae247078be1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 23:31:52 UTC) from {username='slave'} at 127.0.0.1:45378
I20260808 23:31:52.800676 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 7109
I20260808 23:31:52.817392 7719 tablet_replica.cc:330] T ee660a66183d464c95713ae247078be1 P cea7762abf5e4c02abc787a91edbff4d: stopping tablet replica
I20260808 23:31:52.817869 7719 raft_consensus.cc:2229] T ee660a66183d464c95713ae247078be1 P cea7762abf5e4c02abc787a91edbff4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:31:52.818411 7718 tablet_replica.cc:330] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285: stopping tablet replica
I20260808 23:31:52.818442 7719 raft_consensus.cc:2258] T ee660a66183d464c95713ae247078be1 P cea7762abf5e4c02abc787a91edbff4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:31:52.819078 7718 raft_consensus.cc:2229] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 23:31:52.819585 7718 raft_consensus.cc:2258] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:31:52.820456 7717 tablet_replica.cc:330] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37: stopping tablet replica
I20260808 23:31:52.820919 7717 raft_consensus.cc:2229] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37 [term 1 LEADER]: Raft consensus shutting down.
I20260808 23:31:52.821679 7717 raft_consensus.cc:2258] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 23:31:52.822479 7718 ts_tablet_manager.cc:1905] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:31:52.822698 7719 ts_tablet_manager.cc:1905] T ee660a66183d464c95713ae247078be1 P cea7762abf5e4c02abc787a91edbff4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:31:52.823602 7717 ts_tablet_manager.cc:1905] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 23:31:52.828504 7681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:52.802339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55170 (local address 127.9.44.65:39651)
0808 23:31:52.802519 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:52.802526 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:52.812665 (+ 10139us) server_negotiation.cc:407] Connection header received
0808 23:31:52.813061 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:52.813067 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:52.813161 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:52.813231 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:52.813860 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.815025 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.818834 (+ 3809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.819511 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.819630 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:52.826783 (+ 7153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:52.826794 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:52.826809 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:52.827538 (+ 729us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:52.827599 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:52.827625 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:52.827738 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:52.828041 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 23:31:52.828173 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 23:31:52.829700 7718 ts_tablet_manager.cc:1918] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:31:52.829785 7288 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:52.830021 7718 log.cc:1196] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-1/wal/wals/ee660a66183d464c95713ae247078be1
I20260808 23:31:52.830684 7718 ts_tablet_manager.cc:1939] T ee660a66183d464c95713ae247078be1 P 4427eccb16214c1fa4fe914eb1b6f285: Deleting consensus metadata
I20260808 23:31:52.831054 7717 ts_tablet_manager.cc:1918] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 23:31:52.831343 7717 log.cc:1196] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37: Deleting WAL directory at /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786231597224934-9393-0/minicluster-data/ts-2/wal/wals/ee660a66183d464c95713ae247078be1
I20260808 23:31:52.832041 7717 ts_tablet_manager.cc:1939] T ee660a66183d464c95713ae247078be1 P 9edd2328d74645289d916f0ead8bce37: Deleting consensus metadata
I20260808 23:31:52.832554 6999 catalog_manager.cc:4841] TS 4427eccb16214c1fa4fe914eb1b6f285 (127.9.44.66:38897): tablet ee660a66183d464c95713ae247078be1 (table test-table [id=194d25f3fba541c095f309e4ead2810b]) successfully deleted
I20260808 23:31:52.833647 6999 catalog_manager.cc:4841] TS 9edd2328d74645289d916f0ead8bce37 (127.9.44.67:44719): tablet ee660a66183d464c95713ae247078be1 (table test-table [id=194d25f3fba541c095f309e4ead2810b]) successfully deleted
I20260808 23:31:52.969568 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 7109
W20260808 23:31:52.994707 6998 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.44.65:39651 (error 108)
I20260808 23:31:52.995519 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 7302
I20260808 23:31:52.997243 7723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:52.996145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:39651 (local address 127.0.0.1:55172)
0808 23:31:52.996881 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:52.997028 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:39651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":483,"threads_started":1}
W20260808 23:31:52.997536 6998 catalog_manager.cc:4584] TS cea7762abf5e4c02abc787a91edbff4d (127.9.44.65:39651): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ee660a66183d464c95713ae247078be1: Network error: Client connection negotiation failed: client connection to 127.9.44.65:39651: connect: Connection refused (error 111)
I20260808 23:31:53.002772 7682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:52.996307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45394 (local address 127.9.44.66:38897)
0808 23:31:52.996508 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:52.996515 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:52.996826 (+ 311us) server_negotiation.cc:407] Connection header received
0808 23:31:52.996899 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:52.996904 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:52.996997 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:52.997085 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:52.997589 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:52.998403 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:52.999728 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:53.000164 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:53.000287 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:53.001416 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:53.001426 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:53.001440 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:53.002037 (+ 597us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:53.002086 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:53.002114 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:53.002220 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:53.002372 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 23:31:53.002466 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 23:31:53.003388 7476 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:53.051324 7723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:53.050745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:39651 (local address 127.0.0.1:55176)
0808 23:31:53.051002 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:53.051123 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:39651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260808 23:31:53.052556 7723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:53.052133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:39651 (local address 127.0.0.1:55186)
0808 23:31:53.052298 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:53.052386 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:39651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 23:31:53.108716 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 7302
I20260808 23:31:53.118486 7723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:53.117897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:39651 (local address 127.0.0.1:55202)
0808 23:31:53.118153 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:53.118280 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:39651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260808 23:31:53.119920 7723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:53.119393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:39651 (local address 127.0.0.1:55218)
0808 23:31:53.119629 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:53.119747 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:39651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
I20260808 23:31:53.135937 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 7485
I20260808 23:31:53.143499 7678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:53.136736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48082 (local address 127.9.44.67:44719)
0808 23:31:53.136887 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:53.136892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:53.136948 (+ 56us) server_negotiation.cc:407] Connection header received
0808 23:31:53.137178 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:53.137183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:53.137299 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:53.137376 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:53.137878 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:53.138618 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:53.140452 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:53.140908 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:53.140999 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:53.142137 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:53.142146 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:53.142161 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:53.142769 (+ 608us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:53.142833 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:53.142853 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:53.142938 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:53.143123 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 23:31:53.143207 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 23:31:53.144344 7649 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:53.200587 7723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:53.200114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:39651 (local address 127.0.0.1:55220)
0808 23:31:53.200317 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:53.200403 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:39651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 23:31:53.201709 7723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:53.201220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:39651 (local address 127.0.0.1:55226)
0808 23:31:53.201460 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:53.201549 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:39651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260808 23:31:53.263628 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 7485
I20260808 23:31:53.289883 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 6974
I20260808 23:31:53.290966 7070 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:53.397329 7723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:53.396795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.65:39651 (local address 127.0.0.1:55238)
0808 23:31:53.396987 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 23:31:53.397110 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.65:39651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 23:31:53.397312 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 6974
2026-08-08T23:31:53Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3033 ms)
[----------] 6 tests from EncryptionPolicyTest (21823 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:31:53 dist-test-slave-6bbx krb5kdc[7730](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:31:53 dist-test-slave-6bbx krb5kdc[7730](info): set up 2 sockets
Aug 08 23:31:53 dist-test-slave-6bbx krb5kdc[7730](info): commencing operation
krb5kdc: starting...
W20260808 23:31:54.602686 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.003s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:31:54 dist-test-slave-6bbx krb5kdc[7730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:31:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:31:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:31:54.863476 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34617
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:31:55.190867 7746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:55.191167 7746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:55.191314 7746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:55.198189 7746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:31:55.198295 7746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:55.198371 7746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:55.198451 7746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:31:55.198509 7746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:31:55.207926 7746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34617
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:55.209077 7746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:55.210301 7746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:55.221329 7751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:55.221402 7753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:55.221356 7755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:55.221984 7746 server_base.cc:1034] running on GCE node
I20260808 23:31:55.222747 7746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:55.224153 7746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:55.225356 7746 hybrid_clock.cc:648] HybridClock initialized: now 1786231915225293 us; error 47 us; skew 500 ppm
I20260808 23:31:55.225675 7746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:31:55 dist-test-slave-6bbx krb5kdc[7730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231915, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:31:55.228578 7746 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:31:55.229902 7746 webserver.cc:460] Webserver started at http://127.0.0.1:33027/ using document root <none> and password file <none>
I20260808 23:31:55.230536 7746 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:55.230649 7746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:55.230931 7746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:55.233749 7746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "8b2d0c0f484848d9a46f0e6020a72017"
format_stamp: "Formatted at 2026-08-08 23:31:55 on dist-test-slave-6bbx"
server_key: "6a222a9ce71beac3878bad3f654298e6"
server_key_iv: "9ca23abf4d15ac52c312fb55ce59b544"
server_key_version: "encryptionkey@0"
I20260808 23:31:55.234465 7746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "8b2d0c0f484848d9a46f0e6020a72017"
format_stamp: "Formatted at 2026-08-08 23:31:55 on dist-test-slave-6bbx"
server_key: "6a222a9ce71beac3878bad3f654298e6"
server_key_iv: "9ca23abf4d15ac52c312fb55ce59b544"
server_key_version: "encryptionkey@0"
I20260808 23:31:55.239246 7746 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260808 23:31:55.242499 7765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:55.243721 7746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:31:55.243881 7746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "8b2d0c0f484848d9a46f0e6020a72017"
format_stamp: "Formatted at 2026-08-08 23:31:55 on dist-test-slave-6bbx"
server_key: "6a222a9ce71beac3878bad3f654298e6"
server_key_iv: "9ca23abf4d15ac52c312fb55ce59b544"
server_key_version: "encryptionkey@0"
I20260808 23:31:55.244048 7746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:55.259311 7746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:55.262857 7746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:55.263101 7746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:55.280015 7746 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34617
I20260808 23:31:55.280066 7839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34617 every 8 connection(s)
I20260808 23:31:55.281224 7746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:31:55.284677 7841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:55.285645 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 7746
I20260808 23:31:55.285884 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:31:55.286690 9393 external_mini_cluster.cc:1428] Setting key 400800b6cd31c0e9ada187154f68b2cc
I20260808 23:31:55.298184 7841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b2d0c0f484848d9a46f0e6020a72017: Bootstrap starting.
Aug 08 23:31:55 dist-test-slave-6bbx krb5kdc[7730](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 23:31:55.301824 7841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b2d0c0f484848d9a46f0e6020a72017: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:55.302997 7841 log.cc:824] T 00000000000000000000000000000000 P 8b2d0c0f484848d9a46f0e6020a72017: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:55.306078 7841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b2d0c0f484848d9a46f0e6020a72017: No bootstrap required, opened a new log
I20260808 23:31:55.309338 7844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:55.288527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58003 (local address 127.0.0.1:34617)
0808 23:31:55.289298 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:55.289312 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:55.289351 (+ 39us) server_negotiation.cc:407] Connection header received
0808 23:31:55.290936 (+ 1585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:55.290957 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:55.291323 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:55.291626 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:55.293329 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:55.294589 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:55.296427 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:55.297038 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:55.297174 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:55.302036 (+ 4862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:55.302044 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:55.302057 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:55.302094 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:55.305279 (+ 3185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:55.306180 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:55.306188 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:55.306196 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:55.306358 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:55.306846 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:55.306850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:55.306851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:55.307336 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:55.307562 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:55.308090 (+ 528us) server_negotiation.cc:299] Negotiation successful
0808 23:31:55.308410 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":316,"threads_started":1}
I20260808 23:31:55.312973 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:31:55.314489 7841 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b2d0c0f484848d9a46f0e6020a72017 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b2d0c0f484848d9a46f0e6020a72017" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34617 } }
I20260808 23:31:55.314869 7841 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b2d0c0f484848d9a46f0e6020a72017 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:55.314961 7841 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b2d0c0f484848d9a46f0e6020a72017 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2d0c0f484848d9a46f0e6020a72017, State: Initialized, Role: FOLLOWER
I20260808 23:31:55.315582 7841 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b2d0c0f484848d9a46f0e6020a72017 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b2d0c0f484848d9a46f0e6020a72017" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34617 } }
I20260808 23:31:55.315932 7841 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b2d0c0f484848d9a46f0e6020a72017 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:31:55.316072 7841 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b2d0c0f484848d9a46f0e6020a72017 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:31:55.316243 7841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b2d0c0f484848d9a46f0e6020a72017 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:55.318992 7841 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b2d0c0f484848d9a46f0e6020a72017 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b2d0c0f484848d9a46f0e6020a72017" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34617 } }
I20260808 23:31:55.319528 7841 leader_election.cc:302] T 00000000000000000000000000000000 P 8b2d0c0f484848d9a46f0e6020a72017 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b2d0c0f484848d9a46f0e6020a72017; no voters:
I20260808 23:31:55.320132 7841 leader_election.cc:288] T 00000000000000000000000000000000 P 8b2d0c0f484848d9a46f0e6020a72017 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:31:55.320279 7855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b2d0c0f484848d9a46f0e6020a72017 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:31:55.320912 7855 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b2d0c0f484848d9a46f0e6020a72017 [term 1 LEADER]: Becoming Leader. State: Replica: 8b2d0c0f484848d9a46f0e6020a72017, State: Running, Role: LEADER
I20260808 23:31:55.321569 7855 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b2d0c0f484848d9a46f0e6020a72017 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b2d0c0f484848d9a46f0e6020a72017" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34617 } }
I20260808 23:31:55.322675 7841 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b2d0c0f484848d9a46f0e6020a72017 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:31:55.324198 7856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b2d0c0f484848d9a46f0e6020a72017 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b2d0c0f484848d9a46f0e6020a72017" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b2d0c0f484848d9a46f0e6020a72017" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34617 } } }
I20260808 23:31:55.324563 7856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b2d0c0f484848d9a46f0e6020a72017 [sys.catalog]: This master's current role is: LEADER
I20260808 23:31:55.326624 7860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b2d0c0f484848d9a46f0e6020a72017 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b2d0c0f484848d9a46f0e6020a72017. Latest consensus state: current_term: 1 leader_uuid: "8b2d0c0f484848d9a46f0e6020a72017" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b2d0c0f484848d9a46f0e6020a72017" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34617 } } }
I20260808 23:31:55.326925 7860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b2d0c0f484848d9a46f0e6020a72017 [sys.catalog]: This master's current role is: LEADER
I20260808 23:31:55.327320 7865 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:31:55.332450 7865 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:31:55.336050 7844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:55.318407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57231 (local address 127.0.0.1:34617)
0808 23:31:55.318562 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:55.318568 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:55.319056 (+ 488us) server_negotiation.cc:407] Connection header received
0808 23:31:55.319165 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:55.319170 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:55.319238 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:55.319341 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:55.320948 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:55.322106 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:55.323880 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:55.324600 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:55.324733 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:55.327839 (+ 3106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:55.327846 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:55.327852 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:31:55.327890 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:55.331929 (+ 4039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:55.332925 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:55.332930 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:55.332934 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:55.333091 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:31:55.333705 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:31:55.333710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:31:55.333713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:31:55.334116 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:55.335257 (+ 1141us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:55.335466 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 23:31:55.335651 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 23:31:55.341821 7865 catalog_manager.cc:1347] Generated new cluster ID: 690637f29f6a4e22a969f8514d523e70
I20260808 23:31:55.341928 7865 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:31:55.362304 7865 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:31:55.363762 7865 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:31:55.369514 7865 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b2d0c0f484848d9a46f0e6020a72017: Generated new TSK 0
I20260808 23:31:55.370177 7865 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:31:55.375448 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 7746
I20260808 23:31:55.377111 7834 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:31:55.465072 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 7746
W20260808 23:31:55.483245 7728 connection.cc:535] client connection to 127.0.0.1:34617 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34617): Connection reset by peer (error 104)
2026-08-08T23:31:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2057 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:31:55 dist-test-slave-6bbx krb5kdc[7887](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:31:55 dist-test-slave-6bbx krb5kdc[7887](info): set up 2 sockets
Aug 08 23:31:55 dist-test-slave-6bbx krb5kdc[7887](info): commencing operation
krb5kdc: starting...
W20260808 23:31:56.652024 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:31:56 dist-test-slave-6bbx krb5kdc[7887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:31:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:31:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:31:56.920306 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40143
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:31:57.245263 7903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:57.245525 7903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:57.245656 7903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:57.252559 7903 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:31:57.252660 7903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:57.252733 7903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:57.252815 7903 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:31:57.252874 7903 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:31:57.262156 7903 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40143
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:57.263294 7903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:57.264539 7903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:57.275502 7910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:57.275614 7911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:57.275714 7913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:57.276532 7903 server_base.cc:1034] running on GCE node
I20260808 23:31:57.277295 7903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:57.278589 7903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:57.279747 7903 hybrid_clock.cc:648] HybridClock initialized: now 1786231917279701 us; error 31 us; skew 500 ppm
I20260808 23:31:57.280066 7903 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:31:57 dist-test-slave-6bbx krb5kdc[7887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231917, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:31:57.283030 7903 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:31:57.284399 7903 webserver.cc:460] Webserver started at http://127.0.0.1:36327/ using document root <none> and password file <none>
I20260808 23:31:57.285035 7903 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:57.285151 7903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:57.285439 7903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:57.288280 7903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "b702edf75f654cad9e4487db9434ec4a"
format_stamp: "Formatted at 2026-08-08 23:31:57 on dist-test-slave-6bbx"
server_key: "474df7a4314b1913a5387e6c7b2441d7"
server_key_iv: "8e010321ace58f27594f50aae7079d66"
server_key_version: "encryptionkey@0"
I20260808 23:31:57.289006 7903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "b702edf75f654cad9e4487db9434ec4a"
format_stamp: "Formatted at 2026-08-08 23:31:57 on dist-test-slave-6bbx"
server_key: "474df7a4314b1913a5387e6c7b2441d7"
server_key_iv: "8e010321ace58f27594f50aae7079d66"
server_key_version: "encryptionkey@0"
I20260808 23:31:57.293774 7903 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 23:31:57.296985 7925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:57.298177 7903 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260808 23:31:57.298343 7903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "b702edf75f654cad9e4487db9434ec4a"
format_stamp: "Formatted at 2026-08-08 23:31:57 on dist-test-slave-6bbx"
server_key: "474df7a4314b1913a5387e6c7b2441d7"
server_key_iv: "8e010321ace58f27594f50aae7079d66"
server_key_version: "encryptionkey@0"
I20260808 23:31:57.298528 7903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:57.312389 7903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:57.316293 7903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:57.316541 7903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:57.333204 7903 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40143
I20260808 23:31:57.333238 7990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40143 every 8 connection(s)
I20260808 23:31:57.334434 7903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:31:57.336740 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 7903
I20260808 23:31:57.336931 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:31:57.337481 9393 external_mini_cluster.cc:1428] Setting key 6d67dd8e1b6133398f125446510e6bfd
I20260808 23:31:57.338326 7991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:57.349634 7991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b702edf75f654cad9e4487db9434ec4a: Bootstrap starting.
I20260808 23:31:57.349655 7994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:57.339113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50263 (local address 127.0.0.1:40143)
0808 23:31:57.339890 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:57.339901 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:57.339940 (+ 39us) server_negotiation.cc:407] Connection header received
0808 23:31:57.341491 (+ 1551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:57.341512 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:57.341883 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:57.342277 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:57.342848 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:57.344245 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:57.345527 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:57.346162 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:57.346329 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:57.347532 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:57.347540 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:57.347553 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:57.348107 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:57.348178 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:57.348212 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:57.348356 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:57.348664 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 23:31:57.348799 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":321,"threads_started":1}
I20260808 23:31:57.353084 7991 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b702edf75f654cad9e4487db9434ec4a: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:57.353416 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:31:57.354059 7991 log.cc:824] T 00000000000000000000000000000000 P b702edf75f654cad9e4487db9434ec4a: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:57.356598 7991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b702edf75f654cad9e4487db9434ec4a: No bootstrap required, opened a new log
I20260808 23:31:57.363924 7994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:57.357622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39391 (local address 127.0.0.1:40143)
0808 23:31:57.357748 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:57.357752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:57.358078 (+ 326us) server_negotiation.cc:407] Connection header received
0808 23:31:57.358239 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:57.358243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:57.358302 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:57.358389 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:57.358922 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:57.359787 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:57.361563 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:31:57.362025 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:31:57.362133 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:31:57.363208 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:57.363213 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:57.363219 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:57.363234 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:57.363272 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:57.363286 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:57.363366 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:57.363530 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 23:31:57.363571 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 23:31:57.365834 7991 raft_consensus.cc:348] T 00000000000000000000000000000000 P b702edf75f654cad9e4487db9434ec4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b702edf75f654cad9e4487db9434ec4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40143 } }
I20260808 23:31:57.366149 7991 raft_consensus.cc:374] T 00000000000000000000000000000000 P b702edf75f654cad9e4487db9434ec4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:57.366237 7991 raft_consensus.cc:729] T 00000000000000000000000000000000 P b702edf75f654cad9e4487db9434ec4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b702edf75f654cad9e4487db9434ec4a, State: Initialized, Role: FOLLOWER
I20260808 23:31:57.366757 7991 consensus_queue.cc:260] T 00000000000000000000000000000000 P b702edf75f654cad9e4487db9434ec4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b702edf75f654cad9e4487db9434ec4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40143 } }
I20260808 23:31:57.366964 7991 raft_consensus.cc:388] T 00000000000000000000000000000000 P b702edf75f654cad9e4487db9434ec4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:31:57.367089 7991 raft_consensus.cc:482] T 00000000000000000000000000000000 P b702edf75f654cad9e4487db9434ec4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:31:57.367245 7991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b702edf75f654cad9e4487db9434ec4a [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:57.369452 7991 raft_consensus.cc:504] T 00000000000000000000000000000000 P b702edf75f654cad9e4487db9434ec4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b702edf75f654cad9e4487db9434ec4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40143 } }
I20260808 23:31:57.369899 7991 leader_election.cc:302] T 00000000000000000000000000000000 P b702edf75f654cad9e4487db9434ec4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b702edf75f654cad9e4487db9434ec4a; no voters:
I20260808 23:31:57.370404 7991 leader_election.cc:288] T 00000000000000000000000000000000 P b702edf75f654cad9e4487db9434ec4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:31:57.370522 8009 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b702edf75f654cad9e4487db9434ec4a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:31:57.371160 8009 raft_consensus.cc:686] T 00000000000000000000000000000000 P b702edf75f654cad9e4487db9434ec4a [term 1 LEADER]: Becoming Leader. State: Replica: b702edf75f654cad9e4487db9434ec4a, State: Running, Role: LEADER
I20260808 23:31:57.371718 8009 consensus_queue.cc:237] T 00000000000000000000000000000000 P b702edf75f654cad9e4487db9434ec4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b702edf75f654cad9e4487db9434ec4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40143 } }
I20260808 23:31:57.372188 7991 sys_catalog.cc:564] T 00000000000000000000000000000000 P b702edf75f654cad9e4487db9434ec4a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:31:57.374243 8012 sys_catalog.cc:455] T 00000000000000000000000000000000 P b702edf75f654cad9e4487db9434ec4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b702edf75f654cad9e4487db9434ec4a. Latest consensus state: current_term: 1 leader_uuid: "b702edf75f654cad9e4487db9434ec4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b702edf75f654cad9e4487db9434ec4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40143 } } }
I20260808 23:31:57.374688 8012 sys_catalog.cc:458] T 00000000000000000000000000000000 P b702edf75f654cad9e4487db9434ec4a [sys.catalog]: This master's current role is: LEADER
I20260808 23:31:57.374583 8011 sys_catalog.cc:455] T 00000000000000000000000000000000 P b702edf75f654cad9e4487db9434ec4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b702edf75f654cad9e4487db9434ec4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b702edf75f654cad9e4487db9434ec4a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40143 } } }
I20260808 23:31:57.376236 8011 sys_catalog.cc:458] T 00000000000000000000000000000000 P b702edf75f654cad9e4487db9434ec4a [sys.catalog]: This master's current role is: LEADER
I20260808 23:31:57.376963 8020 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:31:57.382004 8020 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:31:57.390344 8020 catalog_manager.cc:1347] Generated new cluster ID: 854ae3effdd74067bcc12dc8a08122a5
I20260808 23:31:57.390434 8020 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:31:57.418139 8020 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:31:57.419407 8020 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:31:57.432309 8020 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b702edf75f654cad9e4487db9434ec4a: Generated new TSK 0
I20260808 23:31:57.433027 8020 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:31:57.448688 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 7903
W20260808 23:31:57.449827 7986 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50263
W20260808 23:31:57.450505 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 7903: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:31:57.450698 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 7903
W20260808 23:31:57.476042 7884 connection.cc:535] client connection to 127.0.0.1:40143 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40143): Connection reset by peer (error 104)
2026-08-08T23:31:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1992 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:31:57 dist-test-slave-6bbx krb5kdc[8041](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:31:57 dist-test-slave-6bbx krb5kdc[8041](info): set up 2 sockets
Aug 08 23:31:57 dist-test-slave-6bbx krb5kdc[8041](info): commencing operation
krb5kdc: starting...
W20260808 23:31:58.653532 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:31:58 dist-test-slave-6bbx krb5kdc[8041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231918, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:31:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:31:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:31:58.918339 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37387
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:31:59.249243 8062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:31:59.249522 8062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:31:59.249624 8062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:31:59.256624 8062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:31:59.256729 8062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:31:59.256781 8062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:31:59.256841 8062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:31:59.256883 8062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:31:59.266378 8062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37387
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:31:59.267524 8062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:31:59.268759 8062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:31:59.280143 8070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:59.280170 8067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:31:59.280103 8068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:31:59.280817 8062 server_base.cc:1034] running on GCE node
I20260808 23:31:59.281563 8062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:31:59.282822 8062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:31:59.284019 8062 hybrid_clock.cc:648] HybridClock initialized: now 1786231919283982 us; error 25 us; skew 500 ppm
I20260808 23:31:59.284340 8062 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:31:59 dist-test-slave-6bbx krb5kdc[8041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231919, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:31:59.290098 8062 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:31:59.291421 8062 webserver.cc:460] Webserver started at http://127.0.0.1:42435/ using document root <none> and password file <none>
I20260808 23:31:59.292105 8062 fs_manager.cc:359] Metadata directory not provided
I20260808 23:31:59.292232 8062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:31:59.292514 8062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:31:59.295334 8062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "e15804730b414e1ebb4f4b76a470fb83"
format_stamp: "Formatted at 2026-08-08 23:31:59 on dist-test-slave-6bbx"
server_key: "56b32018911eacb45603c98d72c1abc2"
server_key_iv: "5c1858d6ea8bb9a27f8f4a53d908ee93"
server_key_version: "encryptionkey@0"
I20260808 23:31:59.296072 8062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "e15804730b414e1ebb4f4b76a470fb83"
format_stamp: "Formatted at 2026-08-08 23:31:59 on dist-test-slave-6bbx"
server_key: "56b32018911eacb45603c98d72c1abc2"
server_key_iv: "5c1858d6ea8bb9a27f8f4a53d908ee93"
server_key_version: "encryptionkey@0"
I20260808 23:31:59.300783 8062 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 23:31:59.304011 8081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:31:59.305140 8062 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260808 23:31:59.305306 8062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "e15804730b414e1ebb4f4b76a470fb83"
format_stamp: "Formatted at 2026-08-08 23:31:59 on dist-test-slave-6bbx"
server_key: "56b32018911eacb45603c98d72c1abc2"
server_key_iv: "5c1858d6ea8bb9a27f8f4a53d908ee93"
server_key_version: "encryptionkey@0"
I20260808 23:31:59.305471 8062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:31:59.310689 8062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:31:59.314659 8062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:31:59.314920 8062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:31:59.331657 8062 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37387
I20260808 23:31:59.331734 8144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37387 every 8 connection(s)
I20260808 23:31:59.332962 8062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:31:59.336508 8146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:31:59.342536 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 8062
I20260808 23:31:59.342693 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:31:59.343134 9393 external_mini_cluster.cc:1428] Setting key 7c990a32bb34869e7c29e3a758eb81e8
I20260808 23:31:59.346625 8146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e15804730b414e1ebb4f4b76a470fb83: Bootstrap starting.
I20260808 23:31:59.349997 8146 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e15804730b414e1ebb4f4b76a470fb83: Neither blocks nor log segments found. Creating new log.
I20260808 23:31:59.350296 8149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:59.344783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47241 (local address 127.0.0.1:37387)
0808 23:31:59.345541 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:59.345552 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:59.345590 (+ 38us) server_negotiation.cc:407] Connection header received
0808 23:31:59.347222 (+ 1632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:59.347245 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:59.347533 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:59.347870 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:59.348888 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:59.348894 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:59.348906 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:59.348920 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:59.348975 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:59.348994 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:59.349144 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:59.349254 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 23:31:59.349401 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":338,"threads_started":1}
I20260808 23:31:59.351255 8146 log.cc:824] T 00000000000000000000000000000000 P e15804730b414e1ebb4f4b76a470fb83: Log is configured to *not* fsync() on all Append() calls
I20260808 23:31:59.353993 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:31:59.354212 8146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e15804730b414e1ebb4f4b76a470fb83: No bootstrap required, opened a new log
I20260808 23:31:59.360347 8149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:31:59.358234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47233 (local address 127.0.0.1:37387)
0808 23:31:59.358401 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 23:31:59.358406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:31:59.358680 (+ 274us) server_negotiation.cc:407] Connection header received
0808 23:31:59.358825 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:31:59.358829 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:31:59.358870 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:31:59.358948 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:31:59.359771 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:31:59.359778 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:31:59.359782 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:31:59.359798 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:31:59.359836 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:31:59.359852 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:31:59.359947 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 23:31:59.360041 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 23:31:59.360080 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 23:31:59.362514 8146 raft_consensus.cc:348] T 00000000000000000000000000000000 P e15804730b414e1ebb4f4b76a470fb83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e15804730b414e1ebb4f4b76a470fb83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37387 } }
I20260808 23:31:59.362835 8146 raft_consensus.cc:374] T 00000000000000000000000000000000 P e15804730b414e1ebb4f4b76a470fb83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:31:59.362936 8146 raft_consensus.cc:729] T 00000000000000000000000000000000 P e15804730b414e1ebb4f4b76a470fb83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e15804730b414e1ebb4f4b76a470fb83, State: Initialized, Role: FOLLOWER
I20260808 23:31:59.363515 8146 consensus_queue.cc:260] T 00000000000000000000000000000000 P e15804730b414e1ebb4f4b76a470fb83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e15804730b414e1ebb4f4b76a470fb83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37387 } }
I20260808 23:31:59.363778 8146 raft_consensus.cc:388] T 00000000000000000000000000000000 P e15804730b414e1ebb4f4b76a470fb83 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:31:59.363921 8146 raft_consensus.cc:482] T 00000000000000000000000000000000 P e15804730b414e1ebb4f4b76a470fb83 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:31:59.364075 8146 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e15804730b414e1ebb4f4b76a470fb83 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:31:59.366222 8146 raft_consensus.cc:504] T 00000000000000000000000000000000 P e15804730b414e1ebb4f4b76a470fb83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e15804730b414e1ebb4f4b76a470fb83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37387 } }
I20260808 23:31:59.366794 8146 leader_election.cc:302] T 00000000000000000000000000000000 P e15804730b414e1ebb4f4b76a470fb83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e15804730b414e1ebb4f4b76a470fb83; no voters:
I20260808 23:31:59.367389 8146 leader_election.cc:288] T 00000000000000000000000000000000 P e15804730b414e1ebb4f4b76a470fb83 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:31:59.367492 8157 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e15804730b414e1ebb4f4b76a470fb83 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:31:59.368180 8157 raft_consensus.cc:686] T 00000000000000000000000000000000 P e15804730b414e1ebb4f4b76a470fb83 [term 1 LEADER]: Becoming Leader. State: Replica: e15804730b414e1ebb4f4b76a470fb83, State: Running, Role: LEADER
I20260808 23:31:59.368645 8157 consensus_queue.cc:237] T 00000000000000000000000000000000 P e15804730b414e1ebb4f4b76a470fb83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e15804730b414e1ebb4f4b76a470fb83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37387 } }
I20260808 23:31:59.369342 8146 sys_catalog.cc:564] T 00000000000000000000000000000000 P e15804730b414e1ebb4f4b76a470fb83 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:31:59.371490 8158 sys_catalog.cc:455] T 00000000000000000000000000000000 P e15804730b414e1ebb4f4b76a470fb83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e15804730b414e1ebb4f4b76a470fb83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e15804730b414e1ebb4f4b76a470fb83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37387 } } }
I20260808 23:31:59.371850 8158 sys_catalog.cc:458] T 00000000000000000000000000000000 P e15804730b414e1ebb4f4b76a470fb83 [sys.catalog]: This master's current role is: LEADER
I20260808 23:31:59.371243 8159 sys_catalog.cc:455] T 00000000000000000000000000000000 P e15804730b414e1ebb4f4b76a470fb83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e15804730b414e1ebb4f4b76a470fb83. Latest consensus state: current_term: 1 leader_uuid: "e15804730b414e1ebb4f4b76a470fb83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e15804730b414e1ebb4f4b76a470fb83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37387 } } }
I20260808 23:31:59.372394 8159 sys_catalog.cc:458] T 00000000000000000000000000000000 P e15804730b414e1ebb4f4b76a470fb83 [sys.catalog]: This master's current role is: LEADER
I20260808 23:31:59.373698 8167 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:31:59.378821 8167 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:31:59.387413 8167 catalog_manager.cc:1347] Generated new cluster ID: 212feee244474c6e93d5050282c9b6c5
I20260808 23:31:59.387517 8167 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:31:59.403070 8167 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:31:59.403528 8167 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:31:59.417889 8167 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e15804730b414e1ebb4f4b76a470fb83: Generated new TSK 0
I20260808 23:31:59.418901 8167 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:31:59.431248 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 8062
W20260808 23:31:59.433079 8140 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47241
W20260808 23:31:59.433725 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 8062: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:31:59.433856 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 8062
2026-08-08T23:31:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1979 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:31:59 dist-test-slave-6bbx krb5kdc[8188](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:31:59 dist-test-slave-6bbx krb5kdc[8188](info): set up 2 sockets
Aug 08 23:31:59 dist-test-slave-6bbx krb5kdc[8188](info): commencing operation
krb5kdc: starting...
W20260808 23:32:00.648559 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:32:00 dist-test-slave-6bbx krb5kdc[8188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:32:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:32:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:32:00.916594 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44897
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:32:01.242990 8207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:32:01.243279 8207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:32:01.243417 8207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:32:01.250362 8207 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:32:01.250479 8207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:32:01.250553 8207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:32:01.250634 8207 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:32:01.250692 8207 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:32:01.260116 8207 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:32:01.261279 8207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:32:01.262526 8207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:32:01.273506 8217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:01.273478 8220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:01.273540 8216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:32:01.274318 8207 server_base.cc:1034] running on GCE node
I20260808 23:32:01.275058 8207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:32:01.276335 8207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:32:01.277539 8207 hybrid_clock.cc:648] HybridClock initialized: now 1786231921277484 us; error 40 us; skew 500 ppm
I20260808 23:32:01.277860 8207 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:32:01 dist-test-slave-6bbx krb5kdc[8188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231921, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:32:01.280805 8207 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:32:01.282155 8207 webserver.cc:460] Webserver started at http://127.0.0.1:43033/ using document root <none> and password file <none>
I20260808 23:32:01.282754 8207 fs_manager.cc:359] Metadata directory not provided
I20260808 23:32:01.282864 8207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:32:01.283100 8207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:32:01.285890 8207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "ee8b11ed500e4bcd9c6146f21a0e8e6d"
format_stamp: "Formatted at 2026-08-08 23:32:01 on dist-test-slave-6bbx"
server_key: "4691d2ac199ae9eea82b2874a3c0885f"
server_key_iv: "3864a7f49f34d4a31e827ffd9893827d"
server_key_version: "encryptionkey@0"
I20260808 23:32:01.286566 8207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "ee8b11ed500e4bcd9c6146f21a0e8e6d"
format_stamp: "Formatted at 2026-08-08 23:32:01 on dist-test-slave-6bbx"
server_key: "4691d2ac199ae9eea82b2874a3c0885f"
server_key_iv: "3864a7f49f34d4a31e827ffd9893827d"
server_key_version: "encryptionkey@0"
I20260808 23:32:01.291185 8207 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 23:32:01.294363 8228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:32:01.295513 8207 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 23:32:01.295660 8207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "ee8b11ed500e4bcd9c6146f21a0e8e6d"
format_stamp: "Formatted at 2026-08-08 23:32:01 on dist-test-slave-6bbx"
server_key: "4691d2ac199ae9eea82b2874a3c0885f"
server_key_iv: "3864a7f49f34d4a31e827ffd9893827d"
server_key_version: "encryptionkey@0"
I20260808 23:32:01.295864 8207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:32:01.320390 8207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:32:01.324281 8207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:32:01.324497 8207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:32:01.341189 8207 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44897
I20260808 23:32:01.341228 8302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44897 every 8 connection(s)
I20260808 23:32:01.342425 8207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:32:01.346011 8304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:32:01.350520 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 8207
I20260808 23:32:01.350687 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:32:01.351073 9393 external_mini_cluster.cc:1428] Setting key 6cbbf88633b0c3c48201025e89eaa275
I20260808 23:32:01.356444 8304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee8b11ed500e4bcd9c6146f21a0e8e6d: Bootstrap starting.
I20260808 23:32:01.359834 8304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee8b11ed500e4bcd9c6146f21a0e8e6d: Neither blocks nor log segments found. Creating new log.
I20260808 23:32:01.360975 8304 log.cc:824] T 00000000000000000000000000000000 P ee8b11ed500e4bcd9c6146f21a0e8e6d: Log is configured to *not* fsync() on all Append() calls
I20260808 23:32:01.364105 8304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee8b11ed500e4bcd9c6146f21a0e8e6d: No bootstrap required, opened a new log
Aug 08 23:32:01 dist-test-slave-6bbx krb5kdc[8188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 23:32:01.372469 8304 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee8b11ed500e4bcd9c6146f21a0e8e6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee8b11ed500e4bcd9c6146f21a0e8e6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44897 } }
I20260808 23:32:01.372833 8304 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee8b11ed500e4bcd9c6146f21a0e8e6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:32:01.372934 8304 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee8b11ed500e4bcd9c6146f21a0e8e6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee8b11ed500e4bcd9c6146f21a0e8e6d, State: Initialized, Role: FOLLOWER
I20260808 23:32:01.373075 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:01.352701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41269 (local address 127.0.0.1:44897)
0808 23:32:01.353455 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:01.353465 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:01.353506 (+ 41us) server_negotiation.cc:407] Connection header received
0808 23:32:01.355193 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:01.355215 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:01.355558 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:01.355904 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:01.357654 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:01.358963 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:01.360701 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:01.361388 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:01.361577 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:01.366192 (+ 4615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:01.366199 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:01.366212 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:01.366247 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:01.369233 (+ 2986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:01.370003 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:01.370008 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:01.370014 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:01.370170 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:01.370538 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:01.370541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:01.370543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:01.371051 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:01.371271 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:01.371746 (+ 475us) server_negotiation.cc:299] Negotiation successful
0808 23:32:01.372093 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":357,"threads_started":1}
I20260808 23:32:01.373569 8304 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee8b11ed500e4bcd9c6146f21a0e8e6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee8b11ed500e4bcd9c6146f21a0e8e6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44897 } }
I20260808 23:32:01.373864 8304 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee8b11ed500e4bcd9c6146f21a0e8e6d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:32:01.374002 8304 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee8b11ed500e4bcd9c6146f21a0e8e6d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:32:01.374188 8304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee8b11ed500e4bcd9c6146f21a0e8e6d [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:32:01.376565 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:32:01.376684 8304 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee8b11ed500e4bcd9c6146f21a0e8e6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee8b11ed500e4bcd9c6146f21a0e8e6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44897 } }
I20260808 23:32:01.377152 8304 leader_election.cc:302] T 00000000000000000000000000000000 P ee8b11ed500e4bcd9c6146f21a0e8e6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee8b11ed500e4bcd9c6146f21a0e8e6d; no voters:
I20260808 23:32:01.377681 8304 leader_election.cc:288] T 00000000000000000000000000000000 P ee8b11ed500e4bcd9c6146f21a0e8e6d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:32:01.377807 8311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee8b11ed500e4bcd9c6146f21a0e8e6d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:32:01.379376 8304 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee8b11ed500e4bcd9c6146f21a0e8e6d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:32:01.383718 8311 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee8b11ed500e4bcd9c6146f21a0e8e6d [term 1 LEADER]: Becoming Leader. State: Replica: ee8b11ed500e4bcd9c6146f21a0e8e6d, State: Running, Role: LEADER
I20260808 23:32:01.384429 8311 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee8b11ed500e4bcd9c6146f21a0e8e6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee8b11ed500e4bcd9c6146f21a0e8e6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44897 } }
I20260808 23:32:01.388968 8312 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee8b11ed500e4bcd9c6146f21a0e8e6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee8b11ed500e4bcd9c6146f21a0e8e6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee8b11ed500e4bcd9c6146f21a0e8e6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44897 } } }
I20260808 23:32:01.389310 8312 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee8b11ed500e4bcd9c6146f21a0e8e6d [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:01.389770 8313 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee8b11ed500e4bcd9c6146f21a0e8e6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee8b11ed500e4bcd9c6146f21a0e8e6d. Latest consensus state: current_term: 1 leader_uuid: "ee8b11ed500e4bcd9c6146f21a0e8e6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee8b11ed500e4bcd9c6146f21a0e8e6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44897 } } }
I20260808 23:32:01.390157 8313 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee8b11ed500e4bcd9c6146f21a0e8e6d [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:01.397933 8335 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:32:01.402372 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:01.384730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44483 (local address 127.0.0.1:44897)
0808 23:32:01.384942 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:01.384947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:01.385455 (+ 508us) server_negotiation.cc:407] Connection header received
0808 23:32:01.385833 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:01.385838 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:01.385917 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:01.386022 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:01.388079 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:01.389300 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:01.391636 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:01.392389 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:01.392495 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:01.395870 (+ 3375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:01.395877 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:01.395883 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:01.395922 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:01.399215 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:01.400132 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:01.400141 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:01.400144 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:01.400288 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:01.400739 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:01.400745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:01.400749 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:01.401173 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:01.401424 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:01.401799 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 23:32:01.402002 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260808 23:32:01.402709 8335 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:32:01.410910 8335 catalog_manager.cc:1347] Generated new cluster ID: bb3a519957ff4e819934da86cd3063e4
I20260808 23:32:01.411000 8335 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:32:01.437282 8335 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:32:01.439023 8335 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:32:01.446244 8335 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee8b11ed500e4bcd9c6146f21a0e8e6d: Generated new TSK 0
I20260808 23:32:01.447141 8335 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:32:01.458799 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 8207
I20260808 23:32:01.460073 8296 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:32:01.549830 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 8207
2026-08-08T23:32:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2117 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:32:01 dist-test-slave-6bbx krb5kdc[8349](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:32:01 dist-test-slave-6bbx krb5kdc[8349](info): set up 2 sockets
Aug 08 23:32:01 dist-test-slave-6bbx krb5kdc[8349](info): commencing operation
krb5kdc: starting...
W20260808 23:32:02.751271 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:32:02 dist-test-slave-6bbx krb5kdc[8349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:32:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:32:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:32:03.035794 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42065
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:32:03.361879 8371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:32:03.362190 8371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:32:03.362332 8371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:32:03.369221 8371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:32:03.369318 8371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:32:03.369392 8371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:32:03.369482 8371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:32:03.369537 8371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:32:03.378721 8371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:32:03.379856 8371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:32:03.381070 8371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:32:03.392181 8377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:03.392145 8378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:03.392221 8381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:32:03.392863 8371 server_base.cc:1034] running on GCE node
I20260808 23:32:03.393559 8371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:32:03.394814 8371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:32:03.396049 8371 hybrid_clock.cc:648] HybridClock initialized: now 1786231923395990 us; error 43 us; skew 500 ppm
I20260808 23:32:03.396392 8371 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:32:03 dist-test-slave-6bbx krb5kdc[8349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231923, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:32:03.399339 8371 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:32:03.400660 8371 webserver.cc:460] Webserver started at http://127.0.0.1:46425/ using document root <none> and password file <none>
I20260808 23:32:03.401296 8371 fs_manager.cc:359] Metadata directory not provided
I20260808 23:32:03.401418 8371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:32:03.401702 8371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:32:03.404569 8371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "150700c3d27349fd8166ce2794bb2023"
format_stamp: "Formatted at 2026-08-08 23:32:03 on dist-test-slave-6bbx"
server_key: "d2a359da4848d720d0b5c944f452c032"
server_key_iv: "9e293c1afee5d5e2027e5c1d5bbd0802"
server_key_version: "encryptionkey@0"
I20260808 23:32:03.405304 8371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "150700c3d27349fd8166ce2794bb2023"
format_stamp: "Formatted at 2026-08-08 23:32:03 on dist-test-slave-6bbx"
server_key: "d2a359da4848d720d0b5c944f452c032"
server_key_iv: "9e293c1afee5d5e2027e5c1d5bbd0802"
server_key_version: "encryptionkey@0"
I20260808 23:32:03.410055 8371 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 23:32:03.413313 8392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:32:03.414520 8371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 23:32:03.414665 8371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "150700c3d27349fd8166ce2794bb2023"
format_stamp: "Formatted at 2026-08-08 23:32:03 on dist-test-slave-6bbx"
server_key: "d2a359da4848d720d0b5c944f452c032"
server_key_iv: "9e293c1afee5d5e2027e5c1d5bbd0802"
server_key_version: "encryptionkey@0"
I20260808 23:32:03.414844 8371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:32:03.427106 8371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:32:03.430549 8371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:32:03.430799 8371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:32:03.449242 8371 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42065
I20260808 23:32:03.449280 8464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42065 every 8 connection(s)
I20260808 23:32:03.450490 8371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:32:03.454015 8466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:32:03.457067 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 8371
I20260808 23:32:03.457264 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:32:03.457821 9393 external_mini_cluster.cc:1428] Setting key f88973f06262fd0afa9fe36ede78ea18
I20260808 23:32:03.466133 8466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 150700c3d27349fd8166ce2794bb2023: Bootstrap starting.
I20260808 23:32:03.469889 8466 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 150700c3d27349fd8166ce2794bb2023: Neither blocks nor log segments found. Creating new log.
I20260808 23:32:03.471064 8473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:03.459461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59357 (local address 127.0.0.1:42065)
0808 23:32:03.460188 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:03.460202 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:03.460248 (+ 46us) server_negotiation.cc:407] Connection header received
0808 23:32:03.461848 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:03.461870 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:03.462219 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:03.462532 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:03.463108 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:03.464449 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:03.466291 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:03.467263 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:03.467460 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:03.468501 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:03.468511 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:03.468526 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:32:03.469176 (+ 650us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:03.469269 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:32:03.469300 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:03.469431 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:03.469581 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 23:32:03.469825 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":334,"threads_started":1}
I20260808 23:32:03.471190 8466 log.cc:824] T 00000000000000000000000000000000 P 150700c3d27349fd8166ce2794bb2023: Log is configured to *not* fsync() on all Append() calls
I20260808 23:32:03.474277 8466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 150700c3d27349fd8166ce2794bb2023: No bootstrap required, opened a new log
I20260808 23:32:03.474843 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:32:03.483843 8466 raft_consensus.cc:348] T 00000000000000000000000000000000 P 150700c3d27349fd8166ce2794bb2023 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150700c3d27349fd8166ce2794bb2023" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42065 } }
I20260808 23:32:03.484301 8466 raft_consensus.cc:374] T 00000000000000000000000000000000 P 150700c3d27349fd8166ce2794bb2023 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:32:03.484419 8466 raft_consensus.cc:729] T 00000000000000000000000000000000 P 150700c3d27349fd8166ce2794bb2023 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 150700c3d27349fd8166ce2794bb2023, State: Initialized, Role: FOLLOWER
I20260808 23:32:03.485038 8466 consensus_queue.cc:260] T 00000000000000000000000000000000 P 150700c3d27349fd8166ce2794bb2023 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150700c3d27349fd8166ce2794bb2023" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42065 } }
I20260808 23:32:03.485301 8466 raft_consensus.cc:388] T 00000000000000000000000000000000 P 150700c3d27349fd8166ce2794bb2023 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:32:03.485455 8466 raft_consensus.cc:482] T 00000000000000000000000000000000 P 150700c3d27349fd8166ce2794bb2023 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:32:03.485627 8466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 150700c3d27349fd8166ce2794bb2023 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:32:03.486129 8473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:03.479975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60399 (local address 127.0.0.1:42065)
0808 23:32:03.480150 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:03.480156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:03.480541 (+ 385us) server_negotiation.cc:407] Connection header received
0808 23:32:03.480674 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:03.480679 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:03.480748 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:03.480856 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:03.481307 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:03.482390 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:03.483623 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:03.484276 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:03.484431 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:03.485429 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:03.485435 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:03.485440 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:32:03.485464 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:03.485504 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:32:03.485519 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:03.485605 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:03.485759 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 23:32:03.485837 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 23:32:03.488247 8466 raft_consensus.cc:504] T 00000000000000000000000000000000 P 150700c3d27349fd8166ce2794bb2023 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150700c3d27349fd8166ce2794bb2023" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42065 } }
I20260808 23:32:03.488706 8466 leader_election.cc:302] T 00000000000000000000000000000000 P 150700c3d27349fd8166ce2794bb2023 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 150700c3d27349fd8166ce2794bb2023; no voters:
I20260808 23:32:03.489207 8466 leader_election.cc:288] T 00000000000000000000000000000000 P 150700c3d27349fd8166ce2794bb2023 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:32:03.489318 8481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 150700c3d27349fd8166ce2794bb2023 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:32:03.489957 8481 raft_consensus.cc:686] T 00000000000000000000000000000000 P 150700c3d27349fd8166ce2794bb2023 [term 1 LEADER]: Becoming Leader. State: Replica: 150700c3d27349fd8166ce2794bb2023, State: Running, Role: LEADER
I20260808 23:32:03.490453 8481 consensus_queue.cc:237] T 00000000000000000000000000000000 P 150700c3d27349fd8166ce2794bb2023 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150700c3d27349fd8166ce2794bb2023" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42065 } }
I20260808 23:32:03.491107 8466 sys_catalog.cc:564] T 00000000000000000000000000000000 P 150700c3d27349fd8166ce2794bb2023 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:32:03.493040 8483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 150700c3d27349fd8166ce2794bb2023 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 150700c3d27349fd8166ce2794bb2023. Latest consensus state: current_term: 1 leader_uuid: "150700c3d27349fd8166ce2794bb2023" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150700c3d27349fd8166ce2794bb2023" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42065 } } }
I20260808 23:32:03.493412 8483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 150700c3d27349fd8166ce2794bb2023 [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:03.494284 8482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 150700c3d27349fd8166ce2794bb2023 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "150700c3d27349fd8166ce2794bb2023" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150700c3d27349fd8166ce2794bb2023" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42065 } } }
I20260808 23:32:03.494534 8482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 150700c3d27349fd8166ce2794bb2023 [sys.catalog]: This master's current role is: LEADER
W20260808 23:32:03.503046 8499 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 150700c3d27349fd8166ce2794bb2023: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:32:03.503162 8499 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:32:03.503280 8489 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:32:03.504899 8489 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:32:03.512312 8489 catalog_manager.cc:1347] Generated new cluster ID: 72ec12bc583a4b27a990c656d70e1846
I20260808 23:32:03.512410 8489 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:32:03.530517 8489 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:32:03.531710 8489 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:32:03.540712 8489 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 150700c3d27349fd8166ce2794bb2023: Generated new TSK 0
I20260808 23:32:03.541407 8489 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:32:03.554843 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 8371
W20260808 23:32:03.555918 8458 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59357
W20260808 23:32:03.556609 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 8371: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:32:03.556754 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 8371
2026-08-08T23:32:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2005 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:32:03 dist-test-slave-6bbx krb5kdc[8508](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:32:03 dist-test-slave-6bbx krb5kdc[8508](info): set up 2 sockets
Aug 08 23:32:03 dist-test-slave-6bbx krb5kdc[8508](info): commencing operation
krb5kdc: starting...
W20260808 23:32:04.773422 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.002s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:32:04 dist-test-slave-6bbx krb5kdc[8508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:32:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:32:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:32:05.039309 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35051
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35051
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:32:05.366894 8526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:32:05.367159 8526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:32:05.367295 8526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:32:05.374190 8526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:32:05.374291 8526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:32:05.374374 8526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:32:05.374454 8526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:32:05.374511 8526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:32:05.383826 8526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35051
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:32:05.384919 8526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:32:05.386138 8526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:32:05.397140 8534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:05.397176 8536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:05.397209 8538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:32:05.397890 8526 server_base.cc:1034] running on GCE node
I20260808 23:32:05.398651 8526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:32:05.399961 8526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:32:05.401157 8526 hybrid_clock.cc:648] HybridClock initialized: now 1786231925401117 us; error 28 us; skew 500 ppm
I20260808 23:32:05.401481 8526 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:32:05 dist-test-slave-6bbx krb5kdc[8508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231925, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:32:05.407222 8526 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:32:05.408639 8526 webserver.cc:460] Webserver started at http://127.0.0.1:43527/ using document root <none> and password file <none>
I20260808 23:32:05.409317 8526 fs_manager.cc:359] Metadata directory not provided
I20260808 23:32:05.409435 8526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:32:05.409725 8526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:32:05.412618 8526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "e9bd38a3389d4724849b0ef5c462559c"
format_stamp: "Formatted at 2026-08-08 23:32:05 on dist-test-slave-6bbx"
server_key: "77ad48c706c0851a38efaec8d8966905"
server_key_iv: "97fab57b22ccdb27fd49d1d05155f991"
server_key_version: "encryptionkey@0"
I20260808 23:32:05.413354 8526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "e9bd38a3389d4724849b0ef5c462559c"
format_stamp: "Formatted at 2026-08-08 23:32:05 on dist-test-slave-6bbx"
server_key: "77ad48c706c0851a38efaec8d8966905"
server_key_iv: "97fab57b22ccdb27fd49d1d05155f991"
server_key_version: "encryptionkey@0"
I20260808 23:32:05.418246 8526 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:32:05.421528 8549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:32:05.422813 8526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 23:32:05.422983 8526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "e9bd38a3389d4724849b0ef5c462559c"
format_stamp: "Formatted at 2026-08-08 23:32:05 on dist-test-slave-6bbx"
server_key: "77ad48c706c0851a38efaec8d8966905"
server_key_iv: "97fab57b22ccdb27fd49d1d05155f991"
server_key_version: "encryptionkey@0"
I20260808 23:32:05.423158 8526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:32:05.428516 8526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:32:05.432142 8526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:32:05.432400 8526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:32:05.449496 8526 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35051
I20260808 23:32:05.449553 8621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35051 every 8 connection(s)
I20260808 23:32:05.450717 8526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:32:05.455796 8622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:32:05.460521 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 8526
I20260808 23:32:05.460691 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:32:05.461103 9393 external_mini_cluster.cc:1428] Setting key 5d8762ed2ceaaf3012c584e2f2bc432f
I20260808 23:32:05.468101 8622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9bd38a3389d4724849b0ef5c462559c: Bootstrap starting.
I20260808 23:32:05.469822 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:05.462713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60983 (local address 127.0.0.1:35051)
0808 23:32:05.463557 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:05.463571 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:05.463624 (+ 53us) server_negotiation.cc:407] Connection header received
0808 23:32:05.466131 (+ 2507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:05.466159 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:05.466683 (+ 524us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:05.467094 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:05.468037 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:05.468045 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:05.468061 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:32:05.468078 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:05.468150 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:32:05.468179 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:05.468386 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:05.468506 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 23:32:05.468691 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":337,"threads_started":1}
I20260808 23:32:05.471810 8622 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9bd38a3389d4724849b0ef5c462559c: Neither blocks nor log segments found. Creating new log.
I20260808 23:32:05.473011 8622 log.cc:824] T 00000000000000000000000000000000 P e9bd38a3389d4724849b0ef5c462559c: Log is configured to *not* fsync() on all Append() calls
I20260808 23:32:05.473424 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:32:05.475968 8622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9bd38a3389d4724849b0ef5c462559c: No bootstrap required, opened a new log
I20260808 23:32:05.480374 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:05.478274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41927 (local address 127.0.0.1:35051)
0808 23:32:05.478433 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:05.478439 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:05.478735 (+ 296us) server_negotiation.cc:407] Connection header received
0808 23:32:05.478862 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:05.478868 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:05.478922 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:05.479034 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:05.479815 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:05.479821 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:05.479829 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:32:05.479847 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:05.479895 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:32:05.479914 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:05.480029 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:05.480074 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 23:32:05.480122 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:32:05.484439 8622 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9bd38a3389d4724849b0ef5c462559c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9bd38a3389d4724849b0ef5c462559c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35051 } }
I20260808 23:32:05.484777 8622 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9bd38a3389d4724849b0ef5c462559c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:32:05.484865 8622 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9bd38a3389d4724849b0ef5c462559c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9bd38a3389d4724849b0ef5c462559c, State: Initialized, Role: FOLLOWER
I20260808 23:32:05.485395 8622 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9bd38a3389d4724849b0ef5c462559c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9bd38a3389d4724849b0ef5c462559c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35051 } }
I20260808 23:32:05.485615 8622 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9bd38a3389d4724849b0ef5c462559c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:32:05.485751 8622 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9bd38a3389d4724849b0ef5c462559c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:32:05.485914 8622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9bd38a3389d4724849b0ef5c462559c [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:32:05.488286 8622 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9bd38a3389d4724849b0ef5c462559c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9bd38a3389d4724849b0ef5c462559c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35051 } }
I20260808 23:32:05.488750 8622 leader_election.cc:302] T 00000000000000000000000000000000 P e9bd38a3389d4724849b0ef5c462559c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9bd38a3389d4724849b0ef5c462559c; no voters:
I20260808 23:32:05.489271 8622 leader_election.cc:288] T 00000000000000000000000000000000 P e9bd38a3389d4724849b0ef5c462559c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:32:05.489389 8635 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9bd38a3389d4724849b0ef5c462559c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:32:05.490302 8635 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9bd38a3389d4724849b0ef5c462559c [term 1 LEADER]: Becoming Leader. State: Replica: e9bd38a3389d4724849b0ef5c462559c, State: Running, Role: LEADER
I20260808 23:32:05.490780 8635 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9bd38a3389d4724849b0ef5c462559c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9bd38a3389d4724849b0ef5c462559c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35051 } }
I20260808 23:32:05.491042 8622 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9bd38a3389d4724849b0ef5c462559c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:32:05.494133 8637 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9bd38a3389d4724849b0ef5c462559c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9bd38a3389d4724849b0ef5c462559c. Latest consensus state: current_term: 1 leader_uuid: "e9bd38a3389d4724849b0ef5c462559c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9bd38a3389d4724849b0ef5c462559c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35051 } } }
I20260808 23:32:05.494534 8637 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9bd38a3389d4724849b0ef5c462559c [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:05.495011 8636 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9bd38a3389d4724849b0ef5c462559c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9bd38a3389d4724849b0ef5c462559c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9bd38a3389d4724849b0ef5c462559c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35051 } } }
I20260808 23:32:05.495329 8636 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9bd38a3389d4724849b0ef5c462559c [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:05.500164 8645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:32:05.504920 8645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:32:05.517819 8645 catalog_manager.cc:1347] Generated new cluster ID: 8a695fc4309a4bb8808a08fe51a6e41b
I20260808 23:32:05.517918 8645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:32:05.552918 8645 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:32:05.553284 8645 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:32:05.566862 8645 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9bd38a3389d4724849b0ef5c462559c: Generated new TSK 0
I20260808 23:32:05.567772 8645 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:32:05.584765 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 8526
W20260808 23:32:05.585819 8614 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60983
W20260808 23:32:05.586618 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 8526: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:32:05.586766 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 8526
2026-08-08T23:32:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2035 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:32:05 dist-test-slave-6bbx krb5kdc[8666](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:32:05 dist-test-slave-6bbx krb5kdc[8666](info): set up 2 sockets
Aug 08 23:32:05 dist-test-slave-6bbx krb5kdc[8666](info): commencing operation
krb5kdc: starting...
W20260808 23:32:06.789728 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.001s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:32:06 dist-test-slave-6bbx krb5kdc[8666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:32:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:32:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:32:07.060392 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45219
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45219
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:32:07.384753 8680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:32:07.385004 8680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:32:07.385121 8680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:32:07.391992 8680 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:32:07.392084 8680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:32:07.392163 8680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:32:07.392242 8680 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:32:07.392300 8680 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:32:07.401463 8680 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:32:07.402590 8680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:32:07.403848 8680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:32:07.414852 8688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:07.414927 8686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:07.414919 8692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:32:07.415633 8680 server_base.cc:1034] running on GCE node
I20260808 23:32:07.416421 8680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:32:07.417678 8680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:32:07.418876 8680 hybrid_clock.cc:648] HybridClock initialized: now 1786231927418829 us; error 40 us; skew 500 ppm
I20260808 23:32:07.419203 8680 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:32:07 dist-test-slave-6bbx krb5kdc[8666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231927, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:32:07.422060 8680 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:32:07.423460 8680 webserver.cc:460] Webserver started at http://127.0.0.1:44279/ using document root <none> and password file <none>
I20260808 23:32:07.424124 8680 fs_manager.cc:359] Metadata directory not provided
I20260808 23:32:07.424249 8680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:32:07.424530 8680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:32:07.427327 8680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "5cc4ccf266f24f96b05eb89631889c35"
format_stamp: "Formatted at 2026-08-08 23:32:07 on dist-test-slave-6bbx"
server_key: "0c718fd09533977428001534a146746d"
server_key_iv: "b507ea2c0aacae83e471ee5302e26905"
server_key_version: "encryptionkey@0"
I20260808 23:32:07.428074 8680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "5cc4ccf266f24f96b05eb89631889c35"
format_stamp: "Formatted at 2026-08-08 23:32:07 on dist-test-slave-6bbx"
server_key: "0c718fd09533977428001534a146746d"
server_key_iv: "b507ea2c0aacae83e471ee5302e26905"
server_key_version: "encryptionkey@0"
I20260808 23:32:07.432770 8680 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 23:32:07.436004 8703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:32:07.437170 8680 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260808 23:32:07.437326 8680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "5cc4ccf266f24f96b05eb89631889c35"
format_stamp: "Formatted at 2026-08-08 23:32:07 on dist-test-slave-6bbx"
server_key: "0c718fd09533977428001534a146746d"
server_key_iv: "b507ea2c0aacae83e471ee5302e26905"
server_key_version: "encryptionkey@0"
I20260808 23:32:07.437492 8680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:32:07.469105 8680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:32:07.472713 8680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:32:07.472966 8680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:32:07.489899 8680 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45219
I20260808 23:32:07.489924 8773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45219 every 8 connection(s)
I20260808 23:32:07.491122 8680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:32:07.493718 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 8680
I20260808 23:32:07.493898 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:32:07.494320 9393 external_mini_cluster.cc:1428] Setting key 265ba5fabf19bd5e022a3f1e8b6c5e47
I20260808 23:32:07.494689 8774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:32:07.508279 8774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cc4ccf266f24f96b05eb89631889c35: Bootstrap starting.
Aug 08 23:32:07 dist-test-slave-6bbx krb5kdc[8666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 23:32:07.511656 8774 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cc4ccf266f24f96b05eb89631889c35: Neither blocks nor log segments found. Creating new log.
I20260808 23:32:07.512987 8774 log.cc:824] T 00000000000000000000000000000000 P 5cc4ccf266f24f96b05eb89631889c35: Log is configured to *not* fsync() on all Append() calls
I20260808 23:32:07.516113 8774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cc4ccf266f24f96b05eb89631889c35: No bootstrap required, opened a new log
I20260808 23:32:07.518281 8777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:07.496195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:45193 (local address 127.0.0.1:45219)
0808 23:32:07.496966 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:07.496980 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:07.497027 (+ 47us) server_negotiation.cc:407] Connection header received
0808 23:32:07.499250 (+ 2223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:07.499279 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:07.499809 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:07.500220 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:07.501452 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:07.503442 (+ 1990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:07.504727 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:07.505763 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:07.505946 (+ 183us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:07.509881 (+ 3935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:07.509894 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:07.509910 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:07.509950 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:07.513665 (+ 3715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:07.514621 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:07.514630 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:07.514637 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:07.514852 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:07.515264 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:07.515269 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:07.515272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:07.515975 (+ 703us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:07.516283 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:07.516689 (+ 406us) server_negotiation.cc:299] Negotiation successful
0808 23:32:07.517009 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":309,"threads_started":1}
I20260808 23:32:07.521972 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:32:07.524628 8774 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cc4ccf266f24f96b05eb89631889c35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc4ccf266f24f96b05eb89631889c35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45219 } }
I20260808 23:32:07.524958 8774 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5cc4ccf266f24f96b05eb89631889c35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:32:07.525049 8774 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cc4ccf266f24f96b05eb89631889c35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc4ccf266f24f96b05eb89631889c35, State: Initialized, Role: FOLLOWER
I20260808 23:32:07.525625 8774 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cc4ccf266f24f96b05eb89631889c35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc4ccf266f24f96b05eb89631889c35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45219 } }
I20260808 23:32:07.525862 8774 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5cc4ccf266f24f96b05eb89631889c35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:32:07.526026 8774 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cc4ccf266f24f96b05eb89631889c35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:32:07.526192 8774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cc4ccf266f24f96b05eb89631889c35 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:32:07.529145 8774 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cc4ccf266f24f96b05eb89631889c35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc4ccf266f24f96b05eb89631889c35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45219 } }
I20260808 23:32:07.529615 8774 leader_election.cc:302] T 00000000000000000000000000000000 P 5cc4ccf266f24f96b05eb89631889c35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5cc4ccf266f24f96b05eb89631889c35; no voters:
I20260808 23:32:07.530270 8774 leader_election.cc:288] T 00000000000000000000000000000000 P 5cc4ccf266f24f96b05eb89631889c35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:32:07.530332 8792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cc4ccf266f24f96b05eb89631889c35 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:32:07.531430 8792 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5cc4ccf266f24f96b05eb89631889c35 [term 1 LEADER]: Becoming Leader. State: Replica: 5cc4ccf266f24f96b05eb89631889c35, State: Running, Role: LEADER
I20260808 23:32:07.532102 8792 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cc4ccf266f24f96b05eb89631889c35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc4ccf266f24f96b05eb89631889c35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45219 } }
I20260808 23:32:07.532274 8774 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cc4ccf266f24f96b05eb89631889c35 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:32:07.537189 8795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cc4ccf266f24f96b05eb89631889c35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cc4ccf266f24f96b05eb89631889c35. Latest consensus state: current_term: 1 leader_uuid: "5cc4ccf266f24f96b05eb89631889c35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc4ccf266f24f96b05eb89631889c35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45219 } } }
I20260808 23:32:07.537583 8795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cc4ccf266f24f96b05eb89631889c35 [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:07.537497 8793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cc4ccf266f24f96b05eb89631889c35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5cc4ccf266f24f96b05eb89631889c35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc4ccf266f24f96b05eb89631889c35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45219 } } }
I20260808 23:32:07.537812 8793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cc4ccf266f24f96b05eb89631889c35 [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:07.541728 8801 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:32:07.543646 8777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:07.526093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:44787 (local address 127.0.0.1:45219)
0808 23:32:07.526258 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:07.526264 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:07.526693 (+ 429us) server_negotiation.cc:407] Connection header received
0808 23:32:07.526868 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:07.526872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:07.526927 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:07.527006 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:07.528755 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:07.529508 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:07.531245 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:07.531931 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:07.532065 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:07.535180 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:07.535187 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:07.535193 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:07.535230 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:07.539065 (+ 3835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:07.540102 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:07.540107 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:07.540110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:07.540484 (+ 374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:07.541074 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:07.541080 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:07.541084 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:07.541667 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:07.542883 (+ 1216us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:07.543067 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 23:32:07.543245 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 23:32:07.547330 8801 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:32:07.556288 8801 catalog_manager.cc:1347] Generated new cluster ID: 1ca5b5dcaec64040a17fe76057d3c38f
I20260808 23:32:07.556388 8801 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:32:07.577737 8801 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:32:07.579419 8801 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:32:07.593096 8801 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5cc4ccf266f24f96b05eb89631889c35: Generated new TSK 0
I20260808 23:32:07.593864 8801 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:32:07.610323 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 8680
I20260808 23:32:07.611500 8768 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:32:07.700416 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 8680
W20260808 23:32:07.723491 8664 connection.cc:535] client connection to 127.0.0.1:45219 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45219): Connection reset by peer (error 104)
2026-08-08T23:32:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2109 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:32:07 dist-test-slave-6bbx krb5kdc[8826](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:32:07 dist-test-slave-6bbx krb5kdc[8826](info): set up 2 sockets
Aug 08 23:32:07 dist-test-slave-6bbx krb5kdc[8826](info): commencing operation
krb5kdc: starting...
W20260808 23:32:08.903599 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:32:09 dist-test-slave-6bbx krb5kdc[8826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:32:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:32:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:32:09.176488 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43839
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:32:09.498580 8844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:32:09.498854 8844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:32:09.499001 8844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:32:09.506023 8844 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:32:09.506138 8844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:32:09.506215 8844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:32:09.506299 8844 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:32:09.506358 8844 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:32:09.515753 8844 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:32:09.516906 8844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:32:09.518172 8844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:32:09.529201 8855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:09.529214 8853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:09.529263 8852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:32:09.530198 8844 server_base.cc:1034] running on GCE node
I20260808 23:32:09.530930 8844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:32:09.532224 8844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:32:09.533420 8844 hybrid_clock.cc:648] HybridClock initialized: now 1786231929533380 us; error 27 us; skew 500 ppm
I20260808 23:32:09.533731 8844 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:32:09 dist-test-slave-6bbx krb5kdc[8826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231929, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:32:09.536643 8844 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:32:09.538007 8844 webserver.cc:460] Webserver started at http://127.0.0.1:45195/ using document root <none> and password file <none>
I20260808 23:32:09.538641 8844 fs_manager.cc:359] Metadata directory not provided
I20260808 23:32:09.538758 8844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:32:09.539039 8844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:32:09.541880 8844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "f40a6668ff494aa5bf92a2fc25e5ac4e"
format_stamp: "Formatted at 2026-08-08 23:32:09 on dist-test-slave-6bbx"
server_key: "541f0616dce4dc7474fc9be1d48a95c1"
server_key_iv: "a2bf0f96c9ba909f88b6ee50e37faa3f"
server_key_version: "encryptionkey@0"
I20260808 23:32:09.542630 8844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "f40a6668ff494aa5bf92a2fc25e5ac4e"
format_stamp: "Formatted at 2026-08-08 23:32:09 on dist-test-slave-6bbx"
server_key: "541f0616dce4dc7474fc9be1d48a95c1"
server_key_iv: "a2bf0f96c9ba909f88b6ee50e37faa3f"
server_key_version: "encryptionkey@0"
I20260808 23:32:09.547554 8844 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 23:32:09.550882 8866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:32:09.552168 8844 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:32:09.552326 8844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "f40a6668ff494aa5bf92a2fc25e5ac4e"
format_stamp: "Formatted at 2026-08-08 23:32:09 on dist-test-slave-6bbx"
server_key: "541f0616dce4dc7474fc9be1d48a95c1"
server_key_iv: "a2bf0f96c9ba909f88b6ee50e37faa3f"
server_key_version: "encryptionkey@0"
I20260808 23:32:09.552507 8844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:32:09.570932 8844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:32:09.574544 8844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:32:09.574785 8844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:32:09.591807 8844 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43839
I20260808 23:32:09.591847 8945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43839 every 8 connection(s)
I20260808 23:32:09.593043 8844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:32:09.596478 8946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:32:09.598730 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 8844
I20260808 23:32:09.598917 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:32:09.599442 9393 external_mini_cluster.cc:1428] Setting key 7e352c3cf6cef65e5ed6b1cbfea0bfeb
I20260808 23:32:09.607453 8946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f40a6668ff494aa5bf92a2fc25e5ac4e: Bootstrap starting.
I20260808 23:32:09.610452 8946 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f40a6668ff494aa5bf92a2fc25e5ac4e: Neither blocks nor log segments found. Creating new log.
I20260808 23:32:09.611544 8946 log.cc:824] T 00000000000000000000000000000000 P f40a6668ff494aa5bf92a2fc25e5ac4e: Log is configured to *not* fsync() on all Append() calls
I20260808 23:32:09.611958 8951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:09.601265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:33835 (local address 127.0.0.1:43839)
0808 23:32:09.602043 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:09.602054 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:09.602095 (+ 41us) server_negotiation.cc:407] Connection header received
0808 23:32:09.603761 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:09.603785 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:09.604150 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:09.604480 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:09.605003 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:09.606301 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:09.607725 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:09.608378 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:09.608543 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:09.609782 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:09.609789 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:09.609802 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:32:09.610309 (+ 507us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:09.610388 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:32:09.610412 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:09.610553 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:09.610852 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 23:32:09.611001 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":328,"threads_started":1}
I20260808 23:32:09.614320 8946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f40a6668ff494aa5bf92a2fc25e5ac4e: No bootstrap required, opened a new log
I20260808 23:32:09.615496 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:32:09.622022 8946 raft_consensus.cc:348] T 00000000000000000000000000000000 P f40a6668ff494aa5bf92a2fc25e5ac4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40a6668ff494aa5bf92a2fc25e5ac4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43839 } }
I20260808 23:32:09.622469 8946 raft_consensus.cc:374] T 00000000000000000000000000000000 P f40a6668ff494aa5bf92a2fc25e5ac4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:32:09.622572 8946 raft_consensus.cc:729] T 00000000000000000000000000000000 P f40a6668ff494aa5bf92a2fc25e5ac4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f40a6668ff494aa5bf92a2fc25e5ac4e, State: Initialized, Role: FOLLOWER
I20260808 23:32:09.623165 8946 consensus_queue.cc:260] T 00000000000000000000000000000000 P f40a6668ff494aa5bf92a2fc25e5ac4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40a6668ff494aa5bf92a2fc25e5ac4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43839 } }
I20260808 23:32:09.623461 8946 raft_consensus.cc:388] T 00000000000000000000000000000000 P f40a6668ff494aa5bf92a2fc25e5ac4e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:32:09.623603 8946 raft_consensus.cc:482] T 00000000000000000000000000000000 P f40a6668ff494aa5bf92a2fc25e5ac4e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:32:09.623780 8946 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f40a6668ff494aa5bf92a2fc25e5ac4e [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:32:09.625852 8951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:09.619461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:58569 (local address 127.0.0.1:43839)
0808 23:32:09.619632 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:09.619639 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:09.620117 (+ 478us) server_negotiation.cc:407] Connection header received
0808 23:32:09.620261 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:09.620265 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:09.620319 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:09.620408 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:09.620919 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:09.621630 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:09.623463 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:09.624057 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:09.624148 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:09.625183 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:09.625188 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:09.625191 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:32:09.625206 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:09.625243 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:32:09.625260 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:09.625340 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:09.625527 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 23:32:09.625564 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 23:32:09.626487 8946 raft_consensus.cc:504] T 00000000000000000000000000000000 P f40a6668ff494aa5bf92a2fc25e5ac4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40a6668ff494aa5bf92a2fc25e5ac4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43839 } }
I20260808 23:32:09.626988 8946 leader_election.cc:302] T 00000000000000000000000000000000 P f40a6668ff494aa5bf92a2fc25e5ac4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f40a6668ff494aa5bf92a2fc25e5ac4e; no voters:
I20260808 23:32:09.627521 8946 leader_election.cc:288] T 00000000000000000000000000000000 P f40a6668ff494aa5bf92a2fc25e5ac4e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:32:09.627626 8959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f40a6668ff494aa5bf92a2fc25e5ac4e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:32:09.628314 8959 raft_consensus.cc:686] T 00000000000000000000000000000000 P f40a6668ff494aa5bf92a2fc25e5ac4e [term 1 LEADER]: Becoming Leader. State: Replica: f40a6668ff494aa5bf92a2fc25e5ac4e, State: Running, Role: LEADER
I20260808 23:32:09.628793 8959 consensus_queue.cc:237] T 00000000000000000000000000000000 P f40a6668ff494aa5bf92a2fc25e5ac4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40a6668ff494aa5bf92a2fc25e5ac4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43839 } }
I20260808 23:32:09.629473 8946 sys_catalog.cc:564] T 00000000000000000000000000000000 P f40a6668ff494aa5bf92a2fc25e5ac4e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:32:09.632064 8960 sys_catalog.cc:455] T 00000000000000000000000000000000 P f40a6668ff494aa5bf92a2fc25e5ac4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f40a6668ff494aa5bf92a2fc25e5ac4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40a6668ff494aa5bf92a2fc25e5ac4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43839 } } }
I20260808 23:32:09.632472 8960 sys_catalog.cc:458] T 00000000000000000000000000000000 P f40a6668ff494aa5bf92a2fc25e5ac4e [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:09.632203 8961 sys_catalog.cc:455] T 00000000000000000000000000000000 P f40a6668ff494aa5bf92a2fc25e5ac4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f40a6668ff494aa5bf92a2fc25e5ac4e. Latest consensus state: current_term: 1 leader_uuid: "f40a6668ff494aa5bf92a2fc25e5ac4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f40a6668ff494aa5bf92a2fc25e5ac4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43839 } } }
I20260808 23:32:09.632628 8961 sys_catalog.cc:458] T 00000000000000000000000000000000 P f40a6668ff494aa5bf92a2fc25e5ac4e [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:09.636925 8974 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:32:09.641913 8974 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:32:09.650369 8974 catalog_manager.cc:1347] Generated new cluster ID: 245b679cb9f54683a1e1003f2d6833a8
I20260808 23:32:09.650463 8974 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:32:09.662488 8974 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:32:09.663578 8974 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:32:09.671895 8974 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f40a6668ff494aa5bf92a2fc25e5ac4e: Generated new TSK 0
I20260808 23:32:09.672614 8974 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:32:09.682792 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 8844
W20260808 23:32:09.683848 8938 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.28:33835
W20260808 23:32:09.684576 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 8844: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:32:09.684706 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 8844
W20260808 23:32:09.708745 8822 connection.cc:535] client connection to 127.0.0.1:43839 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43839): Connection reset by peer (error 104)
2026-08-08T23:32:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1985 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:32:09 dist-test-slave-6bbx krb5kdc[8996](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:32:09 dist-test-slave-6bbx krb5kdc[8996](info): set up 2 sockets
Aug 08 23:32:09 dist-test-slave-6bbx krb5kdc[8996](info): commencing operation
krb5kdc: starting...
W20260808 23:32:10.905296 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.001s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:32:11 dist-test-slave-6bbx krb5kdc[8996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:32:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:32:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:32:11.171509 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35705
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35705
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:32:11.493214 9016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:32:11.493499 9016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:32:11.493640 9016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:32:11.500576 9016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:32:11.500687 9016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:32:11.500765 9016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:32:11.500844 9016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:32:11.500905 9016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:32:11.510236 9016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:32:11.511415 9016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:32:11.512681 9016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:32:11.523792 9022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:11.523725 9023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:11.523768 9028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:32:11.524500 9016 server_base.cc:1034] running on GCE node
I20260808 23:32:11.525236 9016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:32:11.526543 9016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:32:11.527746 9016 hybrid_clock.cc:648] HybridClock initialized: now 1786231931527701 us; error 42 us; skew 500 ppm
I20260808 23:32:11.528055 9016 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:32:11 dist-test-slave-6bbx krb5kdc[8996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231931, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:32:11.533841 9016 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:32:11.535174 9016 webserver.cc:460] Webserver started at http://127.0.0.1:39789/ using document root <none> and password file <none>
I20260808 23:32:11.535853 9016 fs_manager.cc:359] Metadata directory not provided
I20260808 23:32:11.535976 9016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:32:11.536259 9016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:32:11.539086 9016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "863842e41afd4be4a62a6cb8090560a1"
format_stamp: "Formatted at 2026-08-08 23:32:11 on dist-test-slave-6bbx"
server_key: "919ee96e6fbbfc065cb57a50d74f7ac3"
server_key_iv: "3796656cc05835cb5f970caff48862e1"
server_key_version: "encryptionkey@0"
I20260808 23:32:11.539846 9016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "863842e41afd4be4a62a6cb8090560a1"
format_stamp: "Formatted at 2026-08-08 23:32:11 on dist-test-slave-6bbx"
server_key: "919ee96e6fbbfc065cb57a50d74f7ac3"
server_key_iv: "3796656cc05835cb5f970caff48862e1"
server_key_version: "encryptionkey@0"
I20260808 23:32:11.544695 9016 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 23:32:11.547899 9038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:32:11.549090 9016 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 23:32:11.549252 9016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "863842e41afd4be4a62a6cb8090560a1"
format_stamp: "Formatted at 2026-08-08 23:32:11 on dist-test-slave-6bbx"
server_key: "919ee96e6fbbfc065cb57a50d74f7ac3"
server_key_iv: "3796656cc05835cb5f970caff48862e1"
server_key_version: "encryptionkey@0"
I20260808 23:32:11.549428 9016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:32:11.554786 9016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:32:11.558314 9016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:32:11.558575 9016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:32:11.575018 9016 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35705
I20260808 23:32:11.575069 9114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35705 every 8 connection(s)
I20260808 23:32:11.576277 9016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:32:11.579681 9116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:32:11.582856 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 9016
I20260808 23:32:11.583014 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:32:11.583438 9393 external_mini_cluster.cc:1428] Setting key bbb4c3444591d62c769f507afd6550e9
I20260808 23:32:11.591641 9121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:11.585195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:55583 (local address 127.0.0.1:35705)
0808 23:32:11.585967 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:11.585981 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:11.586031 (+ 50us) server_negotiation.cc:407] Connection header received
0808 23:32:11.588039 (+ 2008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:11.588069 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:11.588483 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:11.588914 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:11.589825 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:11.589833 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:11.589850 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:32:11.589888 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:11.589955 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:32:11.589981 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:11.590200 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:11.590368 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 23:32:11.590550 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":305,"threads_started":1}
I20260808 23:32:11.592435 9116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 863842e41afd4be4a62a6cb8090560a1: Bootstrap starting.
I20260808 23:32:11.595278 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:32:11.595993 9116 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 863842e41afd4be4a62a6cb8090560a1: Neither blocks nor log segments found. Creating new log.
I20260808 23:32:11.597164 9116 log.cc:824] T 00000000000000000000000000000000 P 863842e41afd4be4a62a6cb8090560a1: Log is configured to *not* fsync() on all Append() calls
I20260808 23:32:11.600208 9116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 863842e41afd4be4a62a6cb8090560a1: No bootstrap required, opened a new log
I20260808 23:32:11.602445 9121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:11.600248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:43877 (local address 127.0.0.1:35705)
0808 23:32:11.600396 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:11.600402 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:11.600834 (+ 432us) server_negotiation.cc:407] Connection header received
0808 23:32:11.600935 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:11.600939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:11.600990 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:11.601087 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:11.601855 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:11.601868 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:11.601873 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:32:11.601894 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:11.601939 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:32:11.601961 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:11.602074 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:11.602131 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 23:32:11.602187 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 23:32:11.607859 9116 raft_consensus.cc:348] T 00000000000000000000000000000000 P 863842e41afd4be4a62a6cb8090560a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863842e41afd4be4a62a6cb8090560a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35705 } }
I20260808 23:32:11.608174 9116 raft_consensus.cc:374] T 00000000000000000000000000000000 P 863842e41afd4be4a62a6cb8090560a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:32:11.608244 9116 raft_consensus.cc:729] T 00000000000000000000000000000000 P 863842e41afd4be4a62a6cb8090560a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 863842e41afd4be4a62a6cb8090560a1, State: Initialized, Role: FOLLOWER
I20260808 23:32:11.608719 9116 consensus_queue.cc:260] T 00000000000000000000000000000000 P 863842e41afd4be4a62a6cb8090560a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863842e41afd4be4a62a6cb8090560a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35705 } }
I20260808 23:32:11.608915 9116 raft_consensus.cc:388] T 00000000000000000000000000000000 P 863842e41afd4be4a62a6cb8090560a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:32:11.609009 9116 raft_consensus.cc:482] T 00000000000000000000000000000000 P 863842e41afd4be4a62a6cb8090560a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:32:11.609122 9116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 863842e41afd4be4a62a6cb8090560a1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:32:11.611047 9116 raft_consensus.cc:504] T 00000000000000000000000000000000 P 863842e41afd4be4a62a6cb8090560a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863842e41afd4be4a62a6cb8090560a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35705 } }
I20260808 23:32:11.611465 9116 leader_election.cc:302] T 00000000000000000000000000000000 P 863842e41afd4be4a62a6cb8090560a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 863842e41afd4be4a62a6cb8090560a1; no voters:
I20260808 23:32:11.611968 9116 leader_election.cc:288] T 00000000000000000000000000000000 P 863842e41afd4be4a62a6cb8090560a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:32:11.612062 9129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 863842e41afd4be4a62a6cb8090560a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:32:11.612674 9129 raft_consensus.cc:686] T 00000000000000000000000000000000 P 863842e41afd4be4a62a6cb8090560a1 [term 1 LEADER]: Becoming Leader. State: Replica: 863842e41afd4be4a62a6cb8090560a1, State: Running, Role: LEADER
I20260808 23:32:11.613171 9129 consensus_queue.cc:237] T 00000000000000000000000000000000 P 863842e41afd4be4a62a6cb8090560a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863842e41afd4be4a62a6cb8090560a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35705 } }
I20260808 23:32:11.613749 9116 sys_catalog.cc:564] T 00000000000000000000000000000000 P 863842e41afd4be4a62a6cb8090560a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:32:11.615530 9131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863842e41afd4be4a62a6cb8090560a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 863842e41afd4be4a62a6cb8090560a1. Latest consensus state: current_term: 1 leader_uuid: "863842e41afd4be4a62a6cb8090560a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863842e41afd4be4a62a6cb8090560a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35705 } } }
I20260808 23:32:11.615918 9131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863842e41afd4be4a62a6cb8090560a1 [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:11.615881 9130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863842e41afd4be4a62a6cb8090560a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "863842e41afd4be4a62a6cb8090560a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863842e41afd4be4a62a6cb8090560a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35705 } } }
I20260808 23:32:11.616168 9130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863842e41afd4be4a62a6cb8090560a1 [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:11.618289 9136 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:32:11.622959 9136 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:32:11.631141 9136 catalog_manager.cc:1347] Generated new cluster ID: 60d5d2ae24534311b159a3b47978fa9d
I20260808 23:32:11.631242 9136 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:32:11.647035 9136 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:32:11.647379 9136 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:32:11.658391 9136 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 863842e41afd4be4a62a6cb8090560a1: Generated new TSK 0
I20260808 23:32:11.659060 9136 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:32:11.662842 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 9016
W20260808 23:32:11.663941 9110 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.28:55583
W20260808 23:32:11.664565 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 9016: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:32:11.664741 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 9016
2026-08-08T23:32:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1978 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:32:11 dist-test-slave-6bbx krb5kdc[9160](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:32:11 dist-test-slave-6bbx krb5kdc[9160](info): set up 2 sockets
Aug 08 23:32:11 dist-test-slave-6bbx krb5kdc[9160](info): commencing operation
krb5kdc: starting...
W20260808 23:32:12.894258 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:32:13 dist-test-slave-6bbx krb5kdc[9160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:32:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:32:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:32:13.171900 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45463
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45463
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:32:13.493363 9183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:32:13.493626 9183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:32:13.493716 9183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:32:13.500669 9183 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:32:13.500764 9183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:32:13.500815 9183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:32:13.500873 9183 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:32:13.500922 9183 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:32:13.510283 9183 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45463
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:32:13.511396 9183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:32:13.512627 9183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:32:13.523548 9192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:13.523602 9198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:13.523650 9193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:32:13.524812 9183 server_base.cc:1034] running on GCE node
I20260808 23:32:13.525542 9183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:32:13.526767 9183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:32:13.527952 9183 hybrid_clock.cc:648] HybridClock initialized: now 1786231933527914 us; error 29 us; skew 500 ppm
I20260808 23:32:13.528291 9183 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:32:13 dist-test-slave-6bbx krb5kdc[9160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231933, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:32:13.531172 9183 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:32:13.532598 9183 webserver.cc:460] Webserver started at http://127.0.0.1:34111/ using document root <none> and password file <none>
I20260808 23:32:13.533282 9183 fs_manager.cc:359] Metadata directory not provided
I20260808 23:32:13.533406 9183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:32:13.533689 9183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:32:13.536597 9183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "9f9928179f6f4562b38cdfd968f2be7f"
format_stamp: "Formatted at 2026-08-08 23:32:13 on dist-test-slave-6bbx"
server_key: "339cc594a90347a992c36ceb56a564ac"
server_key_iv: "a3192dbac0a0e89ed2aa762db7ec9088"
server_key_version: "encryptionkey@0"
I20260808 23:32:13.537350 9183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "9f9928179f6f4562b38cdfd968f2be7f"
format_stamp: "Formatted at 2026-08-08 23:32:13 on dist-test-slave-6bbx"
server_key: "339cc594a90347a992c36ceb56a564ac"
server_key_iv: "a3192dbac0a0e89ed2aa762db7ec9088"
server_key_version: "encryptionkey@0"
I20260808 23:32:13.542098 9183 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260808 23:32:13.545372 9206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:32:13.546566 9183 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 23:32:13.546717 9183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "9f9928179f6f4562b38cdfd968f2be7f"
format_stamp: "Formatted at 2026-08-08 23:32:13 on dist-test-slave-6bbx"
server_key: "339cc594a90347a992c36ceb56a564ac"
server_key_iv: "a3192dbac0a0e89ed2aa762db7ec9088"
server_key_version: "encryptionkey@0"
I20260808 23:32:13.546885 9183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:32:13.572376 9183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:32:13.578673 9183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:32:13.578936 9183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:32:13.595861 9183 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45463
I20260808 23:32:13.595875 9280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45463 every 8 connection(s)
I20260808 23:32:13.597149 9183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:32:13.600692 9282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:32:13.605336 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 9183
I20260808 23:32:13.605491 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:32:13.605937 9393 external_mini_cluster.cc:1428] Setting key 19b6efbe83296d83b8e946c17c8f4e86
I20260808 23:32:13.611142 9282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f9928179f6f4562b38cdfd968f2be7f: Bootstrap starting.
I20260808 23:32:13.614553 9282 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f9928179f6f4562b38cdfd968f2be7f: Neither blocks nor log segments found. Creating new log.
I20260808 23:32:13.615756 9282 log.cc:824] T 00000000000000000000000000000000 P 9f9928179f6f4562b38cdfd968f2be7f: Log is configured to *not* fsync() on all Append() calls
I20260808 23:32:13.618475 9282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f9928179f6f4562b38cdfd968f2be7f: No bootstrap required, opened a new log
Aug 08 23:32:13 dist-test-slave-6bbx krb5kdc[9160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 23:32:13.627290 9282 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f9928179f6f4562b38cdfd968f2be7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f9928179f6f4562b38cdfd968f2be7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45463 } }
I20260808 23:32:13.627628 9282 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f9928179f6f4562b38cdfd968f2be7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:32:13.627748 9282 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f9928179f6f4562b38cdfd968f2be7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f9928179f6f4562b38cdfd968f2be7f, State: Initialized, Role: FOLLOWER
I20260808 23:32:13.627866 9286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:13.607518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:57463 (local address 127.0.0.1:45463)
0808 23:32:13.608307 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:13.608318 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:13.608357 (+ 39us) server_negotiation.cc:407] Connection header received
0808 23:32:13.610015 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:13.610038 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:13.610401 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:13.610694 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:13.612504 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:13.613821 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:13.615508 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:13.616322 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:13.616526 (+ 204us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:13.620784 (+ 4258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:13.620791 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:13.620804 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:13.620837 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:13.623899 (+ 3062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:13.624731 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:13.624736 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:13.624742 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:13.624896 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:13.625318 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:13.625321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:13.625323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:13.625805 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:13.626038 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:13.626530 (+ 492us) server_negotiation.cc:299] Negotiation successful
0808 23:32:13.626784 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":379,"threads_started":1}
I20260808 23:32:13.628412 9282 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f9928179f6f4562b38cdfd968f2be7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f9928179f6f4562b38cdfd968f2be7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45463 } }
I20260808 23:32:13.628631 9282 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f9928179f6f4562b38cdfd968f2be7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:32:13.628760 9282 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f9928179f6f4562b38cdfd968f2be7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:32:13.628912 9282 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f9928179f6f4562b38cdfd968f2be7f [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:32:13.631162 9282 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f9928179f6f4562b38cdfd968f2be7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f9928179f6f4562b38cdfd968f2be7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45463 } }
I20260808 23:32:13.631372 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:32:13.631704 9282 leader_election.cc:302] T 00000000000000000000000000000000 P 9f9928179f6f4562b38cdfd968f2be7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f9928179f6f4562b38cdfd968f2be7f; no voters:
I20260808 23:32:13.632292 9282 leader_election.cc:288] T 00000000000000000000000000000000 P 9f9928179f6f4562b38cdfd968f2be7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:32:13.632370 9289 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f9928179f6f4562b38cdfd968f2be7f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:32:13.633496 9289 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f9928179f6f4562b38cdfd968f2be7f [term 1 LEADER]: Becoming Leader. State: Replica: 9f9928179f6f4562b38cdfd968f2be7f, State: Running, Role: LEADER
I20260808 23:32:13.633988 9289 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f9928179f6f4562b38cdfd968f2be7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f9928179f6f4562b38cdfd968f2be7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45463 } }
I20260808 23:32:13.634306 9282 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f9928179f6f4562b38cdfd968f2be7f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:32:13.636502 9293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f9928179f6f4562b38cdfd968f2be7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f9928179f6f4562b38cdfd968f2be7f. Latest consensus state: current_term: 1 leader_uuid: "9f9928179f6f4562b38cdfd968f2be7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f9928179f6f4562b38cdfd968f2be7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45463 } } }
I20260808 23:32:13.636620 9291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f9928179f6f4562b38cdfd968f2be7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f9928179f6f4562b38cdfd968f2be7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f9928179f6f4562b38cdfd968f2be7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45463 } } }
I20260808 23:32:13.636895 9293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f9928179f6f4562b38cdfd968f2be7f [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:13.636896 9291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f9928179f6f4562b38cdfd968f2be7f [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:13.652081 9299 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:32:13.656819 9299 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:32:13.672012 9299 catalog_manager.cc:1347] Generated new cluster ID: d6c2deae75504c06b24a8e1e3dfebf78
I20260808 23:32:13.672111 9299 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:32:13.684870 9286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:13.661000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:60615 (local address 127.0.0.1:45463)
0808 23:32:13.663277 (+ 2277us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:13.663284 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:13.668724 (+ 5440us) server_negotiation.cc:407] Connection header received
0808 23:32:13.668912 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:13.668918 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:13.669010 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:13.669129 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:13.671010 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:13.672251 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:13.674016 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:13.674670 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:13.674792 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:13.677638 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:13.677646 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:13.677652 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:13.677688 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:13.681387 (+ 3699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:13.682243 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:13.682248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:13.682252 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:13.682421 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:13.682852 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:13.682858 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:13.682861 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:13.683267 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:13.683544 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:13.684297 (+ 753us) server_negotiation.cc:299] Negotiation successful
0808 23:32:13.684477 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2153}
I20260808 23:32:13.697742 9299 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:32:13.699451 9299 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:32:13.716300 9299 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f9928179f6f4562b38cdfd968f2be7f: Generated new TSK 0
I20260808 23:32:13.717100 9299 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:32:13.728363 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 9183
I20260808 23:32:13.729781 9276 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:32:13.818876 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 9183
W20260808 23:32:13.841516 9158 connection.cc:535] client connection to 127.0.0.1:45463 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45463): Connection reset by peer (error 104)
2026-08-08T23:32:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2154 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:32:13 dist-test-slave-6bbx krb5kdc[9320](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:32:13 dist-test-slave-6bbx krb5kdc[9320](info): set up 2 sockets
Aug 08 23:32:13 dist-test-slave-6bbx krb5kdc[9320](info): commencing operation
krb5kdc: starting...
W20260808 23:32:15.013926 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:32:15 dist-test-slave-6bbx krb5kdc[9320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:32:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:32:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:32:15.292572 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34855
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:32:15.613211 9340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:32:15.613467 9340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:32:15.613596 9340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:32:15.620405 9340 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:32:15.620505 9340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:32:15.620579 9340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:32:15.620661 9340 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:32:15.620721 9340 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:32:15.629850 9340 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34855
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:32:15.630924 9340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:32:15.632138 9340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:32:15.642927 9350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:15.642933 9354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:15.642969 9352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:32:15.643879 9340 server_base.cc:1034] running on GCE node
I20260808 23:32:15.644543 9340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:32:15.645690 9340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:32:15.646883 9340 hybrid_clock.cc:648] HybridClock initialized: now 1786231935646835 us; error 39 us; skew 500 ppm
I20260808 23:32:15.647162 9340 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:32:15 dist-test-slave-6bbx krb5kdc[9320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231935, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:32:15.650034 9340 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:32:15.651279 9340 webserver.cc:460] Webserver started at http://127.0.0.1:33109/ using document root <none> and password file <none>
I20260808 23:32:15.651916 9340 fs_manager.cc:359] Metadata directory not provided
I20260808 23:32:15.652015 9340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:32:15.652249 9340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:32:15.654896 9340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "9be8590f7fbc4b0f9045c2ba683193a3"
format_stamp: "Formatted at 2026-08-08 23:32:15 on dist-test-slave-6bbx"
server_key: "b05b54dbe6ebdf8b5237154d07416b85"
server_key_iv: "993c180b1be6f74d6dcb3cf309ab513a"
server_key_version: "encryptionkey@0"
I20260808 23:32:15.655575 9340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "9be8590f7fbc4b0f9045c2ba683193a3"
format_stamp: "Formatted at 2026-08-08 23:32:15 on dist-test-slave-6bbx"
server_key: "b05b54dbe6ebdf8b5237154d07416b85"
server_key_iv: "993c180b1be6f74d6dcb3cf309ab513a"
server_key_version: "encryptionkey@0"
I20260808 23:32:15.660197 9340 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 23:32:15.663249 9365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:32:15.664428 9340 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 23:32:15.664559 9340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "9be8590f7fbc4b0f9045c2ba683193a3"
format_stamp: "Formatted at 2026-08-08 23:32:15 on dist-test-slave-6bbx"
server_key: "b05b54dbe6ebdf8b5237154d07416b85"
server_key_iv: "993c180b1be6f74d6dcb3cf309ab513a"
server_key_version: "encryptionkey@0"
I20260808 23:32:15.664706 9340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:32:15.687974 9340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:32:15.691460 9340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:32:15.691705 9340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:32:15.708173 9340 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34855
I20260808 23:32:15.708220 9443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34855 every 8 connection(s)
I20260808 23:32:15.709391 9340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:32:15.710947 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 9340
I20260808 23:32:15.711117 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:32:15.711520 9393 external_mini_cluster.cc:1428] Setting key 9a717ef1ccc1f5a1781d3f672d6b41af
I20260808 23:32:15.713255 9444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:32:15.725451 9449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:15.713178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:35177 (local address 127.0.0.1:34855)
0808 23:32:15.713950 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:15.713964 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:15.714012 (+ 48us) server_negotiation.cc:407] Connection header received
0808 23:32:15.716097 (+ 2085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:15.716125 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:15.716608 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:15.717020 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:15.717498 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:15.719440 (+ 1942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:15.720697 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:15.721675 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:15.721849 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:15.722865 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:15.722875 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:15.722891 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:32:15.723577 (+ 686us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:15.723682 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:32:15.723721 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:15.723887 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:15.724099 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 23:32:15.724274 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":308,"threads_started":1}
I20260808 23:32:15.726846 9444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9be8590f7fbc4b0f9045c2ba683193a3: Bootstrap starting.
I20260808 23:32:15.728927 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:32:15.729930 9444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9be8590f7fbc4b0f9045c2ba683193a3: Neither blocks nor log segments found. Creating new log.
I20260808 23:32:15.730844 9444 log.cc:824] T 00000000000000000000000000000000 P 9be8590f7fbc4b0f9045c2ba683193a3: Log is configured to *not* fsync() on all Append() calls
I20260808 23:32:15.733616 9444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9be8590f7fbc4b0f9045c2ba683193a3: No bootstrap required, opened a new log
I20260808 23:32:15.739292 9449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:15.732923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:46381 (local address 127.0.0.1:34855)
0808 23:32:15.733117 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:15.733124 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:15.733516 (+ 392us) server_negotiation.cc:407] Connection header received
0808 23:32:15.733681 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:15.733686 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:15.733751 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:15.733823 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:15.734395 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:15.735118 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:15.736974 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:15.737426 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:15.737519 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:15.738581 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:15.738585 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:15.738588 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:32:15.738605 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:15.738654 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:32:15.738671 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:15.738761 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:15.738946 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 23:32:15.738985 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 23:32:15.742597 9444 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9be8590f7fbc4b0f9045c2ba683193a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be8590f7fbc4b0f9045c2ba683193a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34855 } }
I20260808 23:32:15.742988 9444 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9be8590f7fbc4b0f9045c2ba683193a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:32:15.743083 9444 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9be8590f7fbc4b0f9045c2ba683193a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9be8590f7fbc4b0f9045c2ba683193a3, State: Initialized, Role: FOLLOWER
I20260808 23:32:15.743635 9444 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9be8590f7fbc4b0f9045c2ba683193a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be8590f7fbc4b0f9045c2ba683193a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34855 } }
I20260808 23:32:15.743887 9444 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9be8590f7fbc4b0f9045c2ba683193a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:32:15.744019 9444 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9be8590f7fbc4b0f9045c2ba683193a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:32:15.744179 9444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9be8590f7fbc4b0f9045c2ba683193a3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:32:15.746214 9444 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9be8590f7fbc4b0f9045c2ba683193a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be8590f7fbc4b0f9045c2ba683193a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34855 } }
I20260808 23:32:15.746660 9444 leader_election.cc:302] T 00000000000000000000000000000000 P 9be8590f7fbc4b0f9045c2ba683193a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9be8590f7fbc4b0f9045c2ba683193a3; no voters:
I20260808 23:32:15.747187 9444 leader_election.cc:288] T 00000000000000000000000000000000 P 9be8590f7fbc4b0f9045c2ba683193a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:32:15.747295 9460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9be8590f7fbc4b0f9045c2ba683193a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:32:15.747968 9460 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9be8590f7fbc4b0f9045c2ba683193a3 [term 1 LEADER]: Becoming Leader. State: Replica: 9be8590f7fbc4b0f9045c2ba683193a3, State: Running, Role: LEADER
I20260808 23:32:15.748520 9460 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9be8590f7fbc4b0f9045c2ba683193a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be8590f7fbc4b0f9045c2ba683193a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34855 } }
I20260808 23:32:15.749245 9444 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9be8590f7fbc4b0f9045c2ba683193a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:32:15.751045 9463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9be8590f7fbc4b0f9045c2ba683193a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9be8590f7fbc4b0f9045c2ba683193a3. Latest consensus state: current_term: 1 leader_uuid: "9be8590f7fbc4b0f9045c2ba683193a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be8590f7fbc4b0f9045c2ba683193a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34855 } } }
I20260808 23:32:15.751679 9463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9be8590f7fbc4b0f9045c2ba683193a3 [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:15.751600 9462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9be8590f7fbc4b0f9045c2ba683193a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9be8590f7fbc4b0f9045c2ba683193a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be8590f7fbc4b0f9045c2ba683193a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34855 } } }
I20260808 23:32:15.752373 9462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9be8590f7fbc4b0f9045c2ba683193a3 [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:15.756518 9476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:32:15.760660 9476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:32:15.768503 9476 catalog_manager.cc:1347] Generated new cluster ID: f82a40ce07084de78dc4de5b98c6adf7
I20260808 23:32:15.768590 9476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:32:15.799604 9476 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:32:15.800877 9476 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:32:15.806737 9476 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9be8590f7fbc4b0f9045c2ba683193a3: Generated new TSK 0
I20260808 23:32:15.807466 9476 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:32:15.812184 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 9340
W20260808 23:32:15.813616 9438 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.28:35177
W20260808 23:32:15.814286 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 9340: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:32:15.814445 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 9340
W20260808 23:32:15.840116 9317 connection.cc:535] client connection to 127.0.0.1:34855 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34855): Connection reset by peer (error 104)
2026-08-08T23:32:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1998 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:32:15 dist-test-slave-6bbx krb5kdc[9493](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:32:15 dist-test-slave-6bbx krb5kdc[9493](info): set up 2 sockets
Aug 08 23:32:15 dist-test-slave-6bbx krb5kdc[9493](info): commencing operation
krb5kdc: starting...
W20260808 23:32:17.020955 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.006s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:32:17 dist-test-slave-6bbx krb5kdc[9493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:32:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:32:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 23:32:17.284236 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34695
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:32:17.608181 9512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:32:17.608448 9512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:32:17.608546 9512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:32:17.615439 9512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:32:17.615531 9512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:32:17.615584 9512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:32:17.615643 9512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:32:17.615716 9512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:32:17.624970 9512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34695
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:32:17.626073 9512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:32:17.627360 9512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:32:17.638461 9521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:17.638502 9522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:17.638530 9524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:32:17.639362 9512 server_base.cc:1034] running on GCE node
I20260808 23:32:17.640159 9512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:32:17.641486 9512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:32:17.642735 9512 hybrid_clock.cc:648] HybridClock initialized: now 1786231937642699 us; error 46 us; skew 500 ppm
I20260808 23:32:17.643067 9512 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:32:17 dist-test-slave-6bbx krb5kdc[9493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231937, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:32:17.648701 9512 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 23:32:17.650099 9512 webserver.cc:460] Webserver started at http://127.0.0.1:43447/ using document root <none> and password file <none>
I20260808 23:32:17.650784 9512 fs_manager.cc:359] Metadata directory not provided
I20260808 23:32:17.650908 9512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:32:17.651208 9512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:32:17.654152 9512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "fcda39b4801347339c0fe439c31c8c2f"
format_stamp: "Formatted at 2026-08-08 23:32:17 on dist-test-slave-6bbx"
server_key: "1fb2f5543960211735ffcfae54267660"
server_key_iv: "f3d10986b05e2c6ab291a11f9f09b216"
server_key_version: "encryptionkey@0"
I20260808 23:32:17.654891 9512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "fcda39b4801347339c0fe439c31c8c2f"
format_stamp: "Formatted at 2026-08-08 23:32:17 on dist-test-slave-6bbx"
server_key: "1fb2f5543960211735ffcfae54267660"
server_key_iv: "f3d10986b05e2c6ab291a11f9f09b216"
server_key_version: "encryptionkey@0"
I20260808 23:32:17.659987 9512 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 23:32:17.663249 9538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:32:17.664506 9512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:32:17.664660 9512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "fcda39b4801347339c0fe439c31c8c2f"
format_stamp: "Formatted at 2026-08-08 23:32:17 on dist-test-slave-6bbx"
server_key: "1fb2f5543960211735ffcfae54267660"
server_key_iv: "f3d10986b05e2c6ab291a11f9f09b216"
server_key_version: "encryptionkey@0"
I20260808 23:32:17.664837 9512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:32:17.670449 9512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:32:17.674491 9512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:32:17.674775 9512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:32:17.691880 9512 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34695
I20260808 23:32:17.691926 9606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34695 every 8 connection(s)
I20260808 23:32:17.693207 9512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:32:17.696914 9607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:32:17.698607 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 9512
I20260808 23:32:17.698810 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:32:17.699331 9393 external_mini_cluster.cc:1428] Setting key 3598df7e134a0b3d1fd5e5847e0c5c4a
I20260808 23:32:17.707049 9610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:17.701304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:48575 (local address 127.0.0.1:34695)
0808 23:32:17.702085 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:17.702097 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:17.702159 (+ 62us) server_negotiation.cc:407] Connection header received
0808 23:32:17.703750 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:17.703774 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:17.704123 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:17.704478 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:17.705483 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:17.705490 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:17.705503 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:32:17.705519 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:17.705585 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:32:17.705606 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:17.705811 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:17.705967 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 23:32:17.706162 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":331,"threads_started":1}
I20260808 23:32:17.709829 9607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcda39b4801347339c0fe439c31c8c2f: Bootstrap starting.
I20260808 23:32:17.710518 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:32:17.712747 9607 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcda39b4801347339c0fe439c31c8c2f: Neither blocks nor log segments found. Creating new log.
I20260808 23:32:17.713829 9607 log.cc:824] T 00000000000000000000000000000000 P fcda39b4801347339c0fe439c31c8c2f: Log is configured to *not* fsync() on all Append() calls
I20260808 23:32:17.717025 9607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcda39b4801347339c0fe439c31c8c2f: No bootstrap required, opened a new log
I20260808 23:32:17.717110 9610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:17.714871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:46757 (local address 127.0.0.1:34695)
0808 23:32:17.715032 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:17.715038 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:17.715354 (+ 316us) server_negotiation.cc:407] Connection header received
0808 23:32:17.715467 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:17.715472 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:17.715530 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:17.715659 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:17.716513 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:17.716519 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:17.716524 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:32:17.716542 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:17.716587 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:32:17.716606 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:17.716737 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:17.716784 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 23:32:17.716839 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 23:32:17.724341 9607 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcda39b4801347339c0fe439c31c8c2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcda39b4801347339c0fe439c31c8c2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34695 } }
I20260808 23:32:17.724732 9607 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcda39b4801347339c0fe439c31c8c2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:32:17.724810 9607 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcda39b4801347339c0fe439c31c8c2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcda39b4801347339c0fe439c31c8c2f, State: Initialized, Role: FOLLOWER
I20260808 23:32:17.725337 9607 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcda39b4801347339c0fe439c31c8c2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcda39b4801347339c0fe439c31c8c2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34695 } }
I20260808 23:32:17.725551 9607 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcda39b4801347339c0fe439c31c8c2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:32:17.725651 9607 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcda39b4801347339c0fe439c31c8c2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:32:17.725765 9607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcda39b4801347339c0fe439c31c8c2f [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:32:17.727880 9607 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcda39b4801347339c0fe439c31c8c2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcda39b4801347339c0fe439c31c8c2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34695 } }
I20260808 23:32:17.728344 9607 leader_election.cc:302] T 00000000000000000000000000000000 P fcda39b4801347339c0fe439c31c8c2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcda39b4801347339c0fe439c31c8c2f; no voters:
I20260808 23:32:17.728878 9607 leader_election.cc:288] T 00000000000000000000000000000000 P fcda39b4801347339c0fe439c31c8c2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:32:17.729010 9621 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcda39b4801347339c0fe439c31c8c2f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:32:17.729740 9621 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcda39b4801347339c0fe439c31c8c2f [term 1 LEADER]: Becoming Leader. State: Replica: fcda39b4801347339c0fe439c31c8c2f, State: Running, Role: LEADER
I20260808 23:32:17.730253 9621 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcda39b4801347339c0fe439c31c8c2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcda39b4801347339c0fe439c31c8c2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34695 } }
I20260808 23:32:17.731078 9607 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcda39b4801347339c0fe439c31c8c2f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:32:17.733114 9623 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcda39b4801347339c0fe439c31c8c2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcda39b4801347339c0fe439c31c8c2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcda39b4801347339c0fe439c31c8c2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34695 } } }
I20260808 23:32:17.733569 9623 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcda39b4801347339c0fe439c31c8c2f [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:17.733938 9626 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcda39b4801347339c0fe439c31c8c2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcda39b4801347339c0fe439c31c8c2f. Latest consensus state: current_term: 1 leader_uuid: "fcda39b4801347339c0fe439c31c8c2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcda39b4801347339c0fe439c31c8c2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34695 } } }
I20260808 23:32:17.734292 9626 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcda39b4801347339c0fe439c31c8c2f [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:17.735391 9636 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:32:17.740908 9636 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:32:17.749069 9636 catalog_manager.cc:1347] Generated new cluster ID: a87c540d3401487380a210214a713fb3
I20260808 23:32:17.749181 9636 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:32:17.788918 9636 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:32:17.789340 9636 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:32:17.799526 9636 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcda39b4801347339c0fe439c31c8c2f: Generated new TSK 0
I20260808 23:32:17.800300 9636 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:32:17.808648 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 9512
W20260808 23:32:17.809799 9598 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.28:48575
W20260808 23:32:17.810519 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 9512: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:32:17.810644 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 9512
2026-08-08T23:32:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1995 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:32:17 dist-test-slave-6bbx krb5kdc[9655](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:32:17 dist-test-slave-6bbx krb5kdc[9655](info): set up 2 sockets
Aug 08 23:32:17 dist-test-slave-6bbx krb5kdc[9655](info): commencing operation
krb5kdc: starting...
W20260808 23:32:19.014680 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:32:19 dist-test-slave-6bbx krb5kdc[9655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:32:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:32:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:32:19.276741 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:37841
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:32921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:37841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:32:19.601727 9674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:32:19.601984 9674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:32:19.602110 9674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:32:19.609045 9674 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:32:19.609144 9674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:32:19.609223 9674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:32:19.609309 9674 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:32:19.609377 9674 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:32:19.618670 9674 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:32921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:37841
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:37841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:32:19.619800 9674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:32:19.620998 9674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:32:19.631906 9687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:19.632071 9684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:19.631937 9685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:32:19.632642 9674 server_base.cc:1034] running on GCE node
I20260808 23:32:19.633317 9674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:32:19.634521 9674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:32:19.635736 9674 hybrid_clock.cc:648] HybridClock initialized: now 1786231939635678 us; error 44 us; skew 500 ppm
I20260808 23:32:19.636059 9674 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:32:19 dist-test-slave-6bbx krb5kdc[9655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231939, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:32:19.638873 9674 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:32:19.640224 9674 webserver.cc:460] Webserver started at http://127.9.44.126:35619/ using document root <none> and password file <none>
I20260808 23:32:19.640836 9674 fs_manager.cc:359] Metadata directory not provided
I20260808 23:32:19.640964 9674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:32:19.641237 9674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:32:19.644030 9674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "e4715817a4514aa790be571e5ee3d1ce"
format_stamp: "Formatted at 2026-08-08 23:32:19 on dist-test-slave-6bbx"
server_key: "ade85f8de26fd7de434d88ccc24519d4"
server_key_iv: "85bfa3fafbad0e19d7de20adf708adba"
server_key_version: "encryptionkey@0"
I20260808 23:32:19.644742 9674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "e4715817a4514aa790be571e5ee3d1ce"
format_stamp: "Formatted at 2026-08-08 23:32:19 on dist-test-slave-6bbx"
server_key: "ade85f8de26fd7de434d88ccc24519d4"
server_key_iv: "85bfa3fafbad0e19d7de20adf708adba"
server_key_version: "encryptionkey@0"
I20260808 23:32:19.649598 9674 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:32:19.652824 9695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:32:19.653977 9674 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 23:32:19.654156 9674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "e4715817a4514aa790be571e5ee3d1ce"
format_stamp: "Formatted at 2026-08-08 23:32:19 on dist-test-slave-6bbx"
server_key: "ade85f8de26fd7de434d88ccc24519d4"
server_key_iv: "85bfa3fafbad0e19d7de20adf708adba"
server_key_version: "encryptionkey@0"
I20260808 23:32:19.654323 9674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:32:19.681331 9674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:32:19.684886 9674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:32:19.685132 9674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:32:19.702005 9674 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:37841
I20260808 23:32:19.702028 9767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:37841 every 8 connection(s)
I20260808 23:32:19.703241 9674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:32:19.706781 9771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:32:19.709849 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 9674
I20260808 23:32:19.710026 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:32:19.710567 9393 external_mini_cluster.cc:1428] Setting key 87c275a7c845fdf46967a2e6e86f33fe
I20260808 23:32:19.718403 9771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4715817a4514aa790be571e5ee3d1ce: Bootstrap starting.
I20260808 23:32:19.721901 9771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4715817a4514aa790be571e5ee3d1ce: Neither blocks nor log segments found. Creating new log.
I20260808 23:32:19.723147 9771 log.cc:824] T 00000000000000000000000000000000 P e4715817a4514aa790be571e5ee3d1ce: Log is configured to *not* fsync() on all Append() calls
Aug 08 23:32:19 dist-test-slave-6bbx krb5kdc[9655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:32:19.725963 9771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4715817a4514aa790be571e5ee3d1ce: No bootstrap required, opened a new log
I20260808 23:32:19.733845 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:19.712296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60703 (local address 127.9.44.126:37841)
0808 23:32:19.713023 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:19.713033 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:19.713072 (+ 39us) server_negotiation.cc:407] Connection header received
0808 23:32:19.714665 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:19.714687 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:19.715065 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:19.715366 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:19.717073 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:19.718348 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:19.720103 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:19.720745 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:19.720868 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:19.725460 (+ 4592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:19.725473 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:19.725492 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:19.725535 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:19.729413 (+ 3878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:19.730174 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:19.730187 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:19.730195 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:19.730416 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:19.730808 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:19.730813 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:19.730817 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:19.731517 (+ 700us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:19.731898 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:19.732225 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 23:32:19.732556 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":331,"threads_started":1}
I20260808 23:32:19.736259 9771 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4715817a4514aa790be571e5ee3d1ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4715817a4514aa790be571e5ee3d1ce" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37841 } }
I20260808 23:32:19.736647 9771 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4715817a4514aa790be571e5ee3d1ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:32:19.736747 9771 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4715817a4514aa790be571e5ee3d1ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4715817a4514aa790be571e5ee3d1ce, State: Initialized, Role: FOLLOWER
I20260808 23:32:19.737377 9771 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4715817a4514aa790be571e5ee3d1ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4715817a4514aa790be571e5ee3d1ce" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37841 } }
I20260808 23:32:19.737594 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:32:19.737619 9771 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4715817a4514aa790be571e5ee3d1ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:32:19.737752 9771 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4715817a4514aa790be571e5ee3d1ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:32:19.737900 9771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4715817a4514aa790be571e5ee3d1ce [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:32:19.739977 9771 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4715817a4514aa790be571e5ee3d1ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4715817a4514aa790be571e5ee3d1ce" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37841 } }
I20260808 23:32:19.740449 9771 leader_election.cc:302] T 00000000000000000000000000000000 P e4715817a4514aa790be571e5ee3d1ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4715817a4514aa790be571e5ee3d1ce; no voters:
I20260808 23:32:19.740975 9771 leader_election.cc:288] T 00000000000000000000000000000000 P e4715817a4514aa790be571e5ee3d1ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:32:19.741106 9784 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4715817a4514aa790be571e5ee3d1ce [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:32:19.742274 9784 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4715817a4514aa790be571e5ee3d1ce [term 1 LEADER]: Becoming Leader. State: Replica: e4715817a4514aa790be571e5ee3d1ce, State: Running, Role: LEADER
I20260808 23:32:19.742872 9784 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4715817a4514aa790be571e5ee3d1ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4715817a4514aa790be571e5ee3d1ce" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37841 } }
I20260808 23:32:19.743201 9771 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4715817a4514aa790be571e5ee3d1ce [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:32:19.745352 9787 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4715817a4514aa790be571e5ee3d1ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4715817a4514aa790be571e5ee3d1ce. Latest consensus state: current_term: 1 leader_uuid: "e4715817a4514aa790be571e5ee3d1ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4715817a4514aa790be571e5ee3d1ce" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37841 } } }
I20260808 23:32:19.745911 9786 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4715817a4514aa790be571e5ee3d1ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4715817a4514aa790be571e5ee3d1ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4715817a4514aa790be571e5ee3d1ce" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 37841 } } }
I20260808 23:32:19.746230 9786 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4715817a4514aa790be571e5ee3d1ce [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:19.746625 9787 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4715817a4514aa790be571e5ee3d1ce [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:19.750120 9794 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:32:19.756003 9794 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:32:19.760951 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:19.742174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46155 (local address 127.9.44.126:37841)
0808 23:32:19.742855 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:19.742875 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:19.742895 (+ 20us) server_negotiation.cc:407] Connection header received
0808 23:32:19.743200 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:19.743205 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:19.743278 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:19.743379 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:19.745255 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:19.746597 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:19.748752 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:19.749473 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:19.749569 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:19.752935 (+ 3366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:19.752943 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:19.752948 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:19.752993 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:19.757126 (+ 4133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:19.758157 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:19.758163 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:19.758175 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:19.758318 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:19.758988 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:19.758994 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:19.758997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:19.759426 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:19.759725 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:19.760404 (+ 679us) server_negotiation.cc:299] Negotiation successful
0808 23:32:19.760578 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563}
I20260808 23:32:19.766331 9794 catalog_manager.cc:1347] Generated new cluster ID: e3ec7e469e4c4d418cacd76a1894cbbd
I20260808 23:32:19.766433 9794 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:32:19.801791 9794 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:32:19.803006 9794 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:32:19.816526 9794 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4715817a4514aa790be571e5ee3d1ce: Generated new TSK 0
I20260808 23:32:19.817240 9794 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:32:19.835105 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 9674
I20260808 23:32:19.836807 9762 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:32:19.930011 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 9674
W20260808 23:32:19.951694 9653 connection.cc:535] client connection to 127.9.44.126:37841 recv error: Network error: failed to read from TLS socket (remote: 127.9.44.126:37841): Connection reset by peer (error 104)
2026-08-08T23:32:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2116 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:32:20 dist-test-slave-6bbx krb5kdc[9815](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:32:20 dist-test-slave-6bbx krb5kdc[9815](info): set up 2 sockets
Aug 08 23:32:20 dist-test-slave-6bbx krb5kdc[9815](info): commencing operation
krb5kdc: starting...
W20260808 23:32:21.135517 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.005s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:32:21 dist-test-slave-6bbx krb5kdc[9815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:32:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:32:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:32:21.397996 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:44599
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:36105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:44599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:32:21.721129 9834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:32:21.721412 9834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:32:21.721553 9834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:32:21.728488 9834 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:32:21.728603 9834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:32:21.728683 9834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:32:21.728770 9834 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:32:21.728832 9834 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:32:21.738139 9834 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:36105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:44599
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:44599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:32:21.739302 9834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:32:21.740599 9834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:32:21.751914 9841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:21.752015 9840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:21.751981 9844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:32:21.752782 9834 server_base.cc:1034] running on GCE node
I20260808 23:32:21.753491 9834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:32:21.754772 9834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:32:21.755963 9834 hybrid_clock.cc:648] HybridClock initialized: now 1786231941755912 us; error 45 us; skew 500 ppm
I20260808 23:32:21.756282 9834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:32:21 dist-test-slave-6bbx krb5kdc[9815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231941, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:32:21.759177 9834 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:32:21.760509 9834 webserver.cc:460] Webserver started at http://127.9.44.126:36285/ using document root <none> and password file <none>
I20260808 23:32:21.761109 9834 fs_manager.cc:359] Metadata directory not provided
I20260808 23:32:21.761209 9834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:32:21.761489 9834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:32:21.764393 9834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "7b7591cf42c646a790553e4aa881b095"
format_stamp: "Formatted at 2026-08-08 23:32:21 on dist-test-slave-6bbx"
server_key: "8dc212f939e8387c2d2c4ddb43a6799f"
server_key_iv: "48252f4637362f3a5267cf94bde218dc"
server_key_version: "encryptionkey@0"
I20260808 23:32:21.765103 9834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "7b7591cf42c646a790553e4aa881b095"
format_stamp: "Formatted at 2026-08-08 23:32:21 on dist-test-slave-6bbx"
server_key: "8dc212f939e8387c2d2c4ddb43a6799f"
server_key_iv: "48252f4637362f3a5267cf94bde218dc"
server_key_version: "encryptionkey@0"
I20260808 23:32:21.769908 9834 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 23:32:21.773110 9856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:32:21.774291 9834 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260808 23:32:21.774441 9834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "7b7591cf42c646a790553e4aa881b095"
format_stamp: "Formatted at 2026-08-08 23:32:21 on dist-test-slave-6bbx"
server_key: "8dc212f939e8387c2d2c4ddb43a6799f"
server_key_iv: "48252f4637362f3a5267cf94bde218dc"
server_key_version: "encryptionkey@0"
I20260808 23:32:21.774614 9834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:32:21.810062 9834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:32:21.813840 9834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:32:21.814082 9834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:32:21.830693 9834 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:44599
I20260808 23:32:21.830729 9929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:44599 every 8 connection(s)
I20260808 23:32:21.831964 9834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:32:21.832388 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 9834
I20260808 23:32:21.832576 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:32:21.833168 9393 external_mini_cluster.cc:1428] Setting key a7e838d313c21256070667f1698c53b5
I20260808 23:32:21.836479 9930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:32:21.847517 9933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:21.834858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46779 (local address 127.9.44.126:44599)
0808 23:32:21.835656 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:21.835703 (+ 47us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:21.835754 (+ 51us) server_negotiation.cc:407] Connection header received
0808 23:32:21.837955 (+ 2201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:21.837984 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:21.838439 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:21.838848 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:21.839389 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:21.841356 (+ 1967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:21.842638 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:21.843653 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:21.843869 (+ 216us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:21.844929 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:21.844941 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:21.844957 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:32:21.845621 (+ 664us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:21.845714 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:32:21.845754 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:21.845931 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:21.846111 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 23:32:21.846283 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":317,"threads_started":1}
I20260808 23:32:21.849623 9930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b7591cf42c646a790553e4aa881b095: Bootstrap starting.
I20260808 23:32:21.850975 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:32:21.852741 9930 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b7591cf42c646a790553e4aa881b095: Neither blocks nor log segments found. Creating new log.
I20260808 23:32:21.853677 9930 log.cc:824] T 00000000000000000000000000000000 P 7b7591cf42c646a790553e4aa881b095: Log is configured to *not* fsync() on all Append() calls
I20260808 23:32:21.856608 9930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b7591cf42c646a790553e4aa881b095: No bootstrap required, opened a new log
I20260808 23:32:21.861215 9933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:21.855135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52713 (local address 127.9.44.126:44599)
0808 23:32:21.855286 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:21.855291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:21.855619 (+ 328us) server_negotiation.cc:407] Connection header received
0808 23:32:21.855780 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:21.855786 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:21.855869 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:21.855971 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:21.856411 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:21.857505 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:21.858728 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:21.859399 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:21.859547 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:21.860527 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:21.860534 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:21.860539 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:32:21.860557 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:21.860598 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:32:21.860616 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:21.860720 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:21.860844 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 23:32:21.860908 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 23:32:21.864846 9930 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b7591cf42c646a790553e4aa881b095 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7591cf42c646a790553e4aa881b095" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44599 } }
I20260808 23:32:21.865160 9930 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b7591cf42c646a790553e4aa881b095 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:32:21.865236 9930 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b7591cf42c646a790553e4aa881b095 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b7591cf42c646a790553e4aa881b095, State: Initialized, Role: FOLLOWER
I20260808 23:32:21.865689 9930 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b7591cf42c646a790553e4aa881b095 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7591cf42c646a790553e4aa881b095" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44599 } }
I20260808 23:32:21.865895 9930 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b7591cf42c646a790553e4aa881b095 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:32:21.865986 9930 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b7591cf42c646a790553e4aa881b095 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:32:21.866111 9930 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b7591cf42c646a790553e4aa881b095 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:32:21.868228 9930 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b7591cf42c646a790553e4aa881b095 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7591cf42c646a790553e4aa881b095" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44599 } }
I20260808 23:32:21.868739 9930 leader_election.cc:302] T 00000000000000000000000000000000 P 7b7591cf42c646a790553e4aa881b095 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b7591cf42c646a790553e4aa881b095; no voters:
I20260808 23:32:21.869306 9930 leader_election.cc:288] T 00000000000000000000000000000000 P 7b7591cf42c646a790553e4aa881b095 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:32:21.869418 9941 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b7591cf42c646a790553e4aa881b095 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:32:21.870136 9941 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b7591cf42c646a790553e4aa881b095 [term 1 LEADER]: Becoming Leader. State: Replica: 7b7591cf42c646a790553e4aa881b095, State: Running, Role: LEADER
I20260808 23:32:21.870656 9941 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b7591cf42c646a790553e4aa881b095 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7591cf42c646a790553e4aa881b095" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44599 } }
I20260808 23:32:21.871203 9930 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b7591cf42c646a790553e4aa881b095 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:32:21.877108 9944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b7591cf42c646a790553e4aa881b095 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b7591cf42c646a790553e4aa881b095. Latest consensus state: current_term: 1 leader_uuid: "7b7591cf42c646a790553e4aa881b095" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7591cf42c646a790553e4aa881b095" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44599 } } }
I20260808 23:32:21.877792 9944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b7591cf42c646a790553e4aa881b095 [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:21.877460 9943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b7591cf42c646a790553e4aa881b095 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b7591cf42c646a790553e4aa881b095" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b7591cf42c646a790553e4aa881b095" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44599 } } }
I20260808 23:32:21.878244 9943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b7591cf42c646a790553e4aa881b095 [sys.catalog]: This master's current role is: LEADER
W20260808 23:32:21.881687 9957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7b7591cf42c646a790553e4aa881b095: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:32:21.881780 9957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:32:21.881894 9959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:32:21.884065 9959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:32:21.892431 9959 catalog_manager.cc:1347] Generated new cluster ID: 8be9c455eddc466dadff5dccf4e18022
I20260808 23:32:21.892532 9959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:32:21.898902 9959 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:32:21.900053 9959 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:32:21.911867 9959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b7591cf42c646a790553e4aa881b095: Generated new TSK 0
I20260808 23:32:21.912571 9959 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:32:21.927371 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 9834
W20260808 23:32:21.928401 9921 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46779
W20260808 23:32:21.929076 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 9834: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:32:21.929236 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 9834
W20260808 23:32:21.953176 9813 connection.cc:535] client connection to 127.9.44.126:44599 recv error: Network error: failed to read from TLS socket (remote: 127.9.44.126:44599): Connection reset by peer (error 104)
2026-08-08T23:32:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2001 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:32:22 dist-test-slave-6bbx krb5kdc[9969](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:32:22 dist-test-slave-6bbx krb5kdc[9969](info): set up 2 sockets
Aug 08 23:32:22 dist-test-slave-6bbx krb5kdc[9969](info): commencing operation
krb5kdc: starting...
W20260808 23:32:23.147509 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.001s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:32:23 dist-test-slave-6bbx krb5kdc[9969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:32:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:32:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:32:23.414192 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:44163
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:44163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:32:23.737442 9983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:32:23.737700 9983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:32:23.737838 9983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:32:23.744767 9983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:32:23.744870 9983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:32:23.744952 9983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:32:23.745036 9983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:32:23.745095 9983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:32:23.754506 9983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:44163
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:44163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:32:23.755653 9983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:32:23.756897 9983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:32:23.768006 9997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:23.768119 9993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:23.768317 9994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:32:23.769076 9983 server_base.cc:1034] running on GCE node
I20260808 23:32:23.769759 9983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:32:23.770982 9983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:32:23.772182 9983 hybrid_clock.cc:648] HybridClock initialized: now 1786231943772133 us; error 40 us; skew 500 ppm
I20260808 23:32:23.772505 9983 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:32:23 dist-test-slave-6bbx krb5kdc[9969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231943, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:32:23.775389 9983 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:32:23.776795 9983 webserver.cc:460] Webserver started at http://127.9.44.126:45611/ using document root <none> and password file <none>
I20260808 23:32:23.777420 9983 fs_manager.cc:359] Metadata directory not provided
I20260808 23:32:23.777537 9983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:32:23.777827 9983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:32:23.780697 9983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "07af349ef4ca4d3b831724ca0b7cc701"
format_stamp: "Formatted at 2026-08-08 23:32:23 on dist-test-slave-6bbx"
server_key: "acdfb957fbfe807d124f0fa5020affdf"
server_key_iv: "671c6c9e9858b65046e3a40ba5b2f821"
server_key_version: "encryptionkey@0"
I20260808 23:32:23.781405 9983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "07af349ef4ca4d3b831724ca0b7cc701"
format_stamp: "Formatted at 2026-08-08 23:32:23 on dist-test-slave-6bbx"
server_key: "acdfb957fbfe807d124f0fa5020affdf"
server_key_iv: "671c6c9e9858b65046e3a40ba5b2f821"
server_key_version: "encryptionkey@0"
I20260808 23:32:23.786222 9983 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 23:32:23.789446 10006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:32:23.790602 9983 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 23:32:23.790779 9983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "07af349ef4ca4d3b831724ca0b7cc701"
format_stamp: "Formatted at 2026-08-08 23:32:23 on dist-test-slave-6bbx"
server_key: "acdfb957fbfe807d124f0fa5020affdf"
server_key_iv: "671c6c9e9858b65046e3a40ba5b2f821"
server_key_version: "encryptionkey@0"
I20260808 23:32:23.790952 9983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:32:23.796298 9983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:32:23.799844 9983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:32:23.800105 9983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:32:23.816767 9983 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:44163
I20260808 23:32:23.816820 10089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:44163 every 8 connection(s)
I20260808 23:32:23.818004 9983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:32:23.821395 10090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:32:23.826711 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 9983
I20260808 23:32:23.826893 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:32:23.827447 9393 external_mini_cluster.cc:1428] Setting key 86f5937dd1d4aa573865258f2820d5f5
I20260808 23:32:23.832033 10090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07af349ef4ca4d3b831724ca0b7cc701: Bootstrap starting.
I20260808 23:32:23.834746 10094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:23.829166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32933 (local address 127.9.44.126:44163)
0808 23:32:23.829967 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:23.829977 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:23.830019 (+ 42us) server_negotiation.cc:407] Connection header received
0808 23:32:23.831686 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:23.831716 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:23.832011 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:23.832341 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:23.833319 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:23.833324 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:23.833337 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:32:23.833357 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:23.833410 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:32:23.833429 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:23.833611 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:23.833718 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 23:32:23.833892 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":380,"threads_started":1}
I20260808 23:32:23.835358 10090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07af349ef4ca4d3b831724ca0b7cc701: Neither blocks nor log segments found. Creating new log.
I20260808 23:32:23.836616 10090 log.cc:824] T 00000000000000000000000000000000 P 07af349ef4ca4d3b831724ca0b7cc701: Log is configured to *not* fsync() on all Append() calls
I20260808 23:32:23.838410 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:32:23.839351 10090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07af349ef4ca4d3b831724ca0b7cc701: No bootstrap required, opened a new log
I20260808 23:32:23.844763 10094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:23.842460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43463 (local address 127.9.44.126:44163)
0808 23:32:23.842594 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:23.842603 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:23.842999 (+ 396us) server_negotiation.cc:407] Connection header received
0808 23:32:23.843150 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:23.843153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:23.843201 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:23.843296 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:23.844152 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:23.844158 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:23.844163 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:32:23.844180 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:23.844225 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:32:23.844239 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:23.844348 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:23.844456 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 23:32:23.844510 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 23:32:23.847211 10090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07af349ef4ca4d3b831724ca0b7cc701 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07af349ef4ca4d3b831724ca0b7cc701" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44163 } }
I20260808 23:32:23.847537 10090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07af349ef4ca4d3b831724ca0b7cc701 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:32:23.847627 10090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07af349ef4ca4d3b831724ca0b7cc701 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07af349ef4ca4d3b831724ca0b7cc701, State: Initialized, Role: FOLLOWER
I20260808 23:32:23.848207 10090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07af349ef4ca4d3b831724ca0b7cc701 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07af349ef4ca4d3b831724ca0b7cc701" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44163 } }
I20260808 23:32:23.848444 10090 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07af349ef4ca4d3b831724ca0b7cc701 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:32:23.848603 10090 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07af349ef4ca4d3b831724ca0b7cc701 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:32:23.848783 10090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07af349ef4ca4d3b831724ca0b7cc701 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:32:23.850812 10090 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07af349ef4ca4d3b831724ca0b7cc701 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07af349ef4ca4d3b831724ca0b7cc701" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44163 } }
I20260808 23:32:23.851262 10090 leader_election.cc:302] T 00000000000000000000000000000000 P 07af349ef4ca4d3b831724ca0b7cc701 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07af349ef4ca4d3b831724ca0b7cc701; no voters:
I20260808 23:32:23.851802 10090 leader_election.cc:288] T 00000000000000000000000000000000 P 07af349ef4ca4d3b831724ca0b7cc701 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:32:23.851890 10103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07af349ef4ca4d3b831724ca0b7cc701 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:32:23.852522 10103 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07af349ef4ca4d3b831724ca0b7cc701 [term 1 LEADER]: Becoming Leader. State: Replica: 07af349ef4ca4d3b831724ca0b7cc701, State: Running, Role: LEADER
I20260808 23:32:23.853056 10103 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07af349ef4ca4d3b831724ca0b7cc701 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07af349ef4ca4d3b831724ca0b7cc701" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44163 } }
I20260808 23:32:23.853768 10090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07af349ef4ca4d3b831724ca0b7cc701 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:32:23.855965 10104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07af349ef4ca4d3b831724ca0b7cc701 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07af349ef4ca4d3b831724ca0b7cc701" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07af349ef4ca4d3b831724ca0b7cc701" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44163 } } }
I20260808 23:32:23.856401 10104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07af349ef4ca4d3b831724ca0b7cc701 [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:23.856217 10107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07af349ef4ca4d3b831724ca0b7cc701 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07af349ef4ca4d3b831724ca0b7cc701. Latest consensus state: current_term: 1 leader_uuid: "07af349ef4ca4d3b831724ca0b7cc701" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07af349ef4ca4d3b831724ca0b7cc701" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44163 } } }
I20260808 23:32:23.857097 10107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07af349ef4ca4d3b831724ca0b7cc701 [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:23.857895 10114 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:32:23.863106 10114 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:32:23.871047 10114 catalog_manager.cc:1347] Generated new cluster ID: 955669f8cf724b97943ec471771e77ea
I20260808 23:32:23.871136 10114 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:32:23.885396 10114 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:32:23.885820 10114 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:32:23.902552 10114 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07af349ef4ca4d3b831724ca0b7cc701: Generated new TSK 0
I20260808 23:32:23.903396 10114 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:32:23.916858 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 9983
W20260808 23:32:23.917865 10084 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:32933
W20260808 23:32:23.918545 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 9983: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:32:23.918680 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 9983
2026-08-08T23:32:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1987 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:32:24 dist-test-slave-6bbx krb5kdc[10134](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:32:24 dist-test-slave-6bbx krb5kdc[10134](info): set up 2 sockets
Aug 08 23:32:24 dist-test-slave-6bbx krb5kdc[10134](info): commencing operation
krb5kdc: starting...
W20260808 23:32:25.124888 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.004s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:32:25 dist-test-slave-6bbx krb5kdc[10134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:32:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:32:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:32:25.390527 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:34131
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:38059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:34131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:32:25.712687 10151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:32:25.712944 10151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:32:25.713040 10151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:32:25.720018 10151 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:32:25.720124 10151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:32:25.720177 10151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:32:25.720237 10151 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:32:25.720278 10151 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:32:25.729591 10151 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:38059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:34131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:34131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:32:25.730695 10151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:32:25.731985 10151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:32:25.743053 10160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:25.743136 10165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:25.743112 10163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:32:25.749008 10151 server_base.cc:1034] running on GCE node
I20260808 23:32:25.749744 10151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:32:25.751050 10151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:32:25.752250 10151 hybrid_clock.cc:648] HybridClock initialized: now 1786231945752194 us; error 43 us; skew 500 ppm
I20260808 23:32:25.752573 10151 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:32:25 dist-test-slave-6bbx krb5kdc[10134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231945, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:32:25.755548 10151 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:32:25.756961 10151 webserver.cc:460] Webserver started at http://127.9.44.126:41643/ using document root <none> and password file <none>
I20260808 23:32:25.757640 10151 fs_manager.cc:359] Metadata directory not provided
I20260808 23:32:25.757742 10151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:32:25.757997 10151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:32:25.760812 10151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "d73bc0c658a348a7837ecd8964c53a08"
format_stamp: "Formatted at 2026-08-08 23:32:25 on dist-test-slave-6bbx"
server_key: "b1daa65f5b274f9377a7c689863da310"
server_key_iv: "7cc5868ad3861df6ec40bc58fa1194ae"
server_key_version: "encryptionkey@0"
I20260808 23:32:25.761516 10151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "d73bc0c658a348a7837ecd8964c53a08"
format_stamp: "Formatted at 2026-08-08 23:32:25 on dist-test-slave-6bbx"
server_key: "b1daa65f5b274f9377a7c689863da310"
server_key_iv: "7cc5868ad3861df6ec40bc58fa1194ae"
server_key_version: "encryptionkey@0"
I20260808 23:32:25.766321 10151 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 23:32:25.769510 10174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:32:25.770694 10151 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 23:32:25.770855 10151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "d73bc0c658a348a7837ecd8964c53a08"
format_stamp: "Formatted at 2026-08-08 23:32:25 on dist-test-slave-6bbx"
server_key: "b1daa65f5b274f9377a7c689863da310"
server_key_iv: "7cc5868ad3861df6ec40bc58fa1194ae"
server_key_version: "encryptionkey@0"
I20260808 23:32:25.771029 10151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:32:25.789911 10151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:32:25.793812 10151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:32:25.794080 10151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:32:25.810869 10151 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:34131
I20260808 23:32:25.810887 10249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:34131 every 8 connection(s)
I20260808 23:32:25.812167 10151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:32:25.813482 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 10151
I20260808 23:32:25.813668 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:32:25.814239 9393 external_mini_cluster.cc:1428] Setting key 9bf08c75710d65b95d8deca3ac17893a
I20260808 23:32:25.816478 10250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 23:32:25 dist-test-slave-6bbx krb5kdc[10134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:32:25.828758 10250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d73bc0c658a348a7837ecd8964c53a08: Bootstrap starting.
I20260808 23:32:25.832688 10250 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d73bc0c658a348a7837ecd8964c53a08: Neither blocks nor log segments found. Creating new log.
I20260808 23:32:25.833848 10250 log.cc:824] T 00000000000000000000000000000000 P d73bc0c658a348a7837ecd8964c53a08: Log is configured to *not* fsync() on all Append() calls
I20260808 23:32:25.837056 10250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d73bc0c658a348a7837ecd8964c53a08: No bootstrap required, opened a new log
I20260808 23:32:25.837250 10253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:25.816015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44033 (local address 127.9.44.126:34131)
0808 23:32:25.816740 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:25.816752 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:25.816794 (+ 42us) server_negotiation.cc:407] Connection header received
0808 23:32:25.818375 (+ 1581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:25.818397 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:25.818792 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:25.819132 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:25.820935 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:25.822267 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:25.823989 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:25.824615 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:25.824746 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:25.829191 (+ 4445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:25.829201 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:25.829218 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:25.829276 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:25.832911 (+ 3635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:25.833556 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:25.833564 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:25.833572 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:25.833799 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:25.834142 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:25.834147 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:25.834150 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:25.834761 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:25.834998 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:25.835501 (+ 503us) server_negotiation.cc:299] Negotiation successful
0808 23:32:25.835944 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":317,"threads_started":1}
I20260808 23:32:25.840839 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:32:25.846292 10250 raft_consensus.cc:348] T 00000000000000000000000000000000 P d73bc0c658a348a7837ecd8964c53a08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d73bc0c658a348a7837ecd8964c53a08" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 34131 } }
I20260808 23:32:25.846685 10250 raft_consensus.cc:374] T 00000000000000000000000000000000 P d73bc0c658a348a7837ecd8964c53a08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:32:25.846802 10250 raft_consensus.cc:729] T 00000000000000000000000000000000 P d73bc0c658a348a7837ecd8964c53a08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d73bc0c658a348a7837ecd8964c53a08, State: Initialized, Role: FOLLOWER
I20260808 23:32:25.847345 10250 consensus_queue.cc:260] T 00000000000000000000000000000000 P d73bc0c658a348a7837ecd8964c53a08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d73bc0c658a348a7837ecd8964c53a08" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 34131 } }
I20260808 23:32:25.847553 10250 raft_consensus.cc:388] T 00000000000000000000000000000000 P d73bc0c658a348a7837ecd8964c53a08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:32:25.847692 10250 raft_consensus.cc:482] T 00000000000000000000000000000000 P d73bc0c658a348a7837ecd8964c53a08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:32:25.847852 10250 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d73bc0c658a348a7837ecd8964c53a08 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:32:25.850416 10250 raft_consensus.cc:504] T 00000000000000000000000000000000 P d73bc0c658a348a7837ecd8964c53a08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d73bc0c658a348a7837ecd8964c53a08" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 34131 } }
I20260808 23:32:25.850927 10250 leader_election.cc:302] T 00000000000000000000000000000000 P d73bc0c658a348a7837ecd8964c53a08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d73bc0c658a348a7837ecd8964c53a08; no voters:
I20260808 23:32:25.851581 10250 leader_election.cc:288] T 00000000000000000000000000000000 P d73bc0c658a348a7837ecd8964c53a08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:32:25.851737 10263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d73bc0c658a348a7837ecd8964c53a08 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:32:25.853516 10250 sys_catalog.cc:564] T 00000000000000000000000000000000 P d73bc0c658a348a7837ecd8964c53a08 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:32:25.854549 10263 raft_consensus.cc:686] T 00000000000000000000000000000000 P d73bc0c658a348a7837ecd8964c53a08 [term 1 LEADER]: Becoming Leader. State: Replica: d73bc0c658a348a7837ecd8964c53a08, State: Running, Role: LEADER
I20260808 23:32:25.855260 10263 consensus_queue.cc:237] T 00000000000000000000000000000000 P d73bc0c658a348a7837ecd8964c53a08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d73bc0c658a348a7837ecd8964c53a08" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 34131 } }
I20260808 23:32:25.858307 10266 sys_catalog.cc:455] T 00000000000000000000000000000000 P d73bc0c658a348a7837ecd8964c53a08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d73bc0c658a348a7837ecd8964c53a08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d73bc0c658a348a7837ecd8964c53a08" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 34131 } } }
I20260808 23:32:25.858677 10266 sys_catalog.cc:458] T 00000000000000000000000000000000 P d73bc0c658a348a7837ecd8964c53a08 [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:25.859216 10267 sys_catalog.cc:455] T 00000000000000000000000000000000 P d73bc0c658a348a7837ecd8964c53a08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d73bc0c658a348a7837ecd8964c53a08. Latest consensus state: current_term: 1 leader_uuid: "d73bc0c658a348a7837ecd8964c53a08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d73bc0c658a348a7837ecd8964c53a08" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 34131 } } }
I20260808 23:32:25.859501 10267 sys_catalog.cc:458] T 00000000000000000000000000000000 P d73bc0c658a348a7837ecd8964c53a08 [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:25.863009 10253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:25.845811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40601 (local address 127.9.44.126:34131)
0808 23:32:25.845973 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:25.845978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:25.846383 (+ 405us) server_negotiation.cc:407] Connection header received
0808 23:32:25.846509 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:25.846513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:25.846585 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:25.846686 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:25.848243 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:25.849365 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:25.850571 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:25.851194 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:25.851318 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:25.854254 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:25.854259 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:25.854265 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:25.854304 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:25.858942 (+ 4638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:25.859867 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:25.859873 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:25.859877 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:25.860080 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:25.860525 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:25.860531 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:25.860534 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:25.860931 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:25.862283 (+ 1352us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:25.862437 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 23:32:25.862649 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 23:32:25.865139 10277 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:32:25.869906 10277 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:32:25.879365 10277 catalog_manager.cc:1347] Generated new cluster ID: 019bccbb9d3e4ddf9ee77468a562db9d
I20260808 23:32:25.879458 10277 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:32:25.895210 10277 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:32:25.897058 10277 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:32:25.909396 10277 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d73bc0c658a348a7837ecd8964c53a08: Generated new TSK 0
I20260808 23:32:25.910149 10277 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:32:25.928138 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 10151
I20260808 23:32:25.929168 10243 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:32:26.018530 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 10151
2026-08-08T23:32:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2098 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:32:26 dist-test-slave-6bbx krb5kdc[10292](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:32:26 dist-test-slave-6bbx krb5kdc[10292](info): set up 2 sockets
Aug 08 23:32:26 dist-test-slave-6bbx krb5kdc[10292](info): commencing operation
krb5kdc: starting...
W20260808 23:32:27.235145 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:32:27 dist-test-slave-6bbx krb5kdc[10292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:32:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:32:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:32:27.500020 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:39457
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:36743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:39457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:32:27.826607 10308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:32:27.826890 10308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:32:27.827023 10308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:32:27.833945 10308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:32:27.834054 10308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:32:27.834127 10308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:32:27.834208 10308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:32:27.834267 10308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:32:27.843623 10308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:36743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:39457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:39457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:32:27.844779 10308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:32:27.846024 10308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:32:27.856886 10318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:27.856976 10315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:27.856932 10316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:32:27.857730 10308 server_base.cc:1034] running on GCE node
I20260808 23:32:27.858459 10308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:32:27.859699 10308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:32:27.860904 10308 hybrid_clock.cc:648] HybridClock initialized: now 1786231947860849 us; error 39 us; skew 500 ppm
I20260808 23:32:27.861220 10308 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:32:27 dist-test-slave-6bbx krb5kdc[10292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231947, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:32:27.864207 10308 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:32:27.865499 10308 webserver.cc:460] Webserver started at http://127.9.44.126:34617/ using document root <none> and password file <none>
I20260808 23:32:27.866106 10308 fs_manager.cc:359] Metadata directory not provided
I20260808 23:32:27.866221 10308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:32:27.866501 10308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:32:27.869269 10308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "45136e528f7c45058720cfe6311d1b86"
format_stamp: "Formatted at 2026-08-08 23:32:27 on dist-test-slave-6bbx"
server_key: "c383487043ec7b0068bd84b7aabdabf1"
server_key_iv: "31c73516b22240c5210c6a2a0c3aa653"
server_key_version: "encryptionkey@0"
I20260808 23:32:27.869967 10308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "45136e528f7c45058720cfe6311d1b86"
format_stamp: "Formatted at 2026-08-08 23:32:27 on dist-test-slave-6bbx"
server_key: "c383487043ec7b0068bd84b7aabdabf1"
server_key_iv: "31c73516b22240c5210c6a2a0c3aa653"
server_key_version: "encryptionkey@0"
I20260808 23:32:27.874663 10308 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260808 23:32:27.877900 10326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:32:27.879070 10308 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 23:32:27.879225 10308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "45136e528f7c45058720cfe6311d1b86"
format_stamp: "Formatted at 2026-08-08 23:32:27 on dist-test-slave-6bbx"
server_key: "c383487043ec7b0068bd84b7aabdabf1"
server_key_iv: "31c73516b22240c5210c6a2a0c3aa653"
server_key_version: "encryptionkey@0"
I20260808 23:32:27.879400 10308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:32:27.892489 10308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:32:27.895932 10308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:32:27.896168 10308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:32:27.912611 10308 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:39457
I20260808 23:32:27.912647 10404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:39457 every 8 connection(s)
I20260808 23:32:27.913842 10308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:32:27.917253 10406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:32:27.919124 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 10308
I20260808 23:32:27.919313 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:32:27.919895 9393 external_mini_cluster.cc:1428] Setting key e9a9625a69c6512a4297ae9d809781db
I20260808 23:32:27.930408 10406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45136e528f7c45058720cfe6311d1b86: Bootstrap starting.
I20260808 23:32:27.932163 10410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:27.921536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44791 (local address 127.9.44.126:39457)
0808 23:32:27.922288 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:27.922298 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:27.922341 (+ 43us) server_negotiation.cc:407] Connection header received
0808 23:32:27.923884 (+ 1543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:27.923908 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:27.924257 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:27.924541 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:27.925092 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:27.926329 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:27.928144 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:27.928784 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:27.928905 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:27.930100 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:27.930107 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:27.930119 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:32:27.930611 (+ 492us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:27.930688 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:32:27.930711 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:27.930802 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:27.931022 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 23:32:27.931245 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":343,"threads_started":1}
I20260808 23:32:27.933542 10406 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45136e528f7c45058720cfe6311d1b86: Neither blocks nor log segments found. Creating new log.
I20260808 23:32:27.934618 10406 log.cc:824] T 00000000000000000000000000000000 P 45136e528f7c45058720cfe6311d1b86: Log is configured to *not* fsync() on all Append() calls
I20260808 23:32:27.935472 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:32:27.937547 10406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45136e528f7c45058720cfe6311d1b86: No bootstrap required, opened a new log
I20260808 23:32:27.945750 10406 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45136e528f7c45058720cfe6311d1b86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45136e528f7c45058720cfe6311d1b86" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39457 } }
I20260808 23:32:27.946183 10406 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45136e528f7c45058720cfe6311d1b86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:32:27.946287 10406 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45136e528f7c45058720cfe6311d1b86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45136e528f7c45058720cfe6311d1b86, State: Initialized, Role: FOLLOWER
I20260808 23:32:27.946918 10406 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45136e528f7c45058720cfe6311d1b86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45136e528f7c45058720cfe6311d1b86" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39457 } }
I20260808 23:32:27.947192 10406 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45136e528f7c45058720cfe6311d1b86 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:32:27.947335 10406 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45136e528f7c45058720cfe6311d1b86 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:32:27.947527 10406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45136e528f7c45058720cfe6311d1b86 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:32:27.949741 10410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:27.940517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35725 (local address 127.9.44.126:39457)
0808 23:32:27.940686 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:27.940692 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:27.944238 (+ 3546us) server_negotiation.cc:407] Connection header received
0808 23:32:27.944403 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:27.944407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:27.944460 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:27.944538 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:27.945081 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:27.945708 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:27.947477 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:27.947987 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:27.948087 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:27.949127 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:27.949132 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:27.949135 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:32:27.949150 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:27.949186 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:32:27.949203 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:27.949280 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:27.949409 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 23:32:27.949474 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 23:32:27.950430 10406 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45136e528f7c45058720cfe6311d1b86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45136e528f7c45058720cfe6311d1b86" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39457 } }
I20260808 23:32:27.950906 10406 leader_election.cc:302] T 00000000000000000000000000000000 P 45136e528f7c45058720cfe6311d1b86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45136e528f7c45058720cfe6311d1b86; no voters:
I20260808 23:32:27.951454 10406 leader_election.cc:288] T 00000000000000000000000000000000 P 45136e528f7c45058720cfe6311d1b86 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:32:27.951555 10420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45136e528f7c45058720cfe6311d1b86 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:32:27.952174 10420 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45136e528f7c45058720cfe6311d1b86 [term 1 LEADER]: Becoming Leader. State: Replica: 45136e528f7c45058720cfe6311d1b86, State: Running, Role: LEADER
I20260808 23:32:27.952656 10420 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45136e528f7c45058720cfe6311d1b86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45136e528f7c45058720cfe6311d1b86" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39457 } }
I20260808 23:32:27.953363 10406 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45136e528f7c45058720cfe6311d1b86 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:32:27.955096 10424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45136e528f7c45058720cfe6311d1b86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45136e528f7c45058720cfe6311d1b86. Latest consensus state: current_term: 1 leader_uuid: "45136e528f7c45058720cfe6311d1b86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45136e528f7c45058720cfe6311d1b86" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39457 } } }
I20260808 23:32:27.955472 10424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45136e528f7c45058720cfe6311d1b86 [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:27.955024 10423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45136e528f7c45058720cfe6311d1b86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45136e528f7c45058720cfe6311d1b86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45136e528f7c45058720cfe6311d1b86" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39457 } } }
I20260808 23:32:27.956005 10423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45136e528f7c45058720cfe6311d1b86 [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:27.958025 10427 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:32:27.963264 10427 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:32:27.970868 10427 catalog_manager.cc:1347] Generated new cluster ID: 9cfa3aacb6274675aaad46b9f0fad005
I20260808 23:32:27.970974 10427 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:32:27.988205 10427 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:32:27.989903 10427 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:32:27.997627 10427 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45136e528f7c45058720cfe6311d1b86: Generated new TSK 0
I20260808 23:32:27.998476 10427 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:32:28.005652 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 10308
W20260808 23:32:28.006656 10399 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44791
W20260808 23:32:28.007282 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 10308: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:32:28.007402 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 10308
2026-08-08T23:32:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1990 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:32:28 dist-test-slave-6bbx krb5kdc[10448](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:32:28 dist-test-slave-6bbx krb5kdc[10448](info): set up 2 sockets
Aug 08 23:32:28 dist-test-slave-6bbx krb5kdc[10448](info): commencing operation
krb5kdc: starting...
W20260808 23:32:29.206115 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.004s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:32:29 dist-test-slave-6bbx krb5kdc[10448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:32:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:32:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:32:29.478258 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:39007
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:37279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:39007
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:32:29.802531 10466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:32:29.802785 10466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:32:29.802912 10466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:32:29.809798 10466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:32:29.809899 10466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:32:29.809975 10466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:32:29.810053 10466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:32:29.810109 10466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:32:29.819380 10466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:37279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:39007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:39007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:32:29.820539 10466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:32:29.821738 10466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:32:29.832775 10474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:29.832841 10478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:29.832849 10472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:32:29.833514 10466 server_base.cc:1034] running on GCE node
I20260808 23:32:29.834254 10466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:32:29.835536 10466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:32:29.836745 10466 hybrid_clock.cc:648] HybridClock initialized: now 1786231949836679 us; error 51 us; skew 500 ppm
I20260808 23:32:29.837069 10466 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:32:29 dist-test-slave-6bbx krb5kdc[10448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231949, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:32:29.840049 10466 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:32:29.841428 10466 webserver.cc:460] Webserver started at http://127.9.44.126:44085/ using document root <none> and password file <none>
I20260808 23:32:29.842052 10466 fs_manager.cc:359] Metadata directory not provided
I20260808 23:32:29.842188 10466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:32:29.842478 10466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:32:29.845357 10466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "3735e2e6880c4f86a96e8f160d079f07"
format_stamp: "Formatted at 2026-08-08 23:32:29 on dist-test-slave-6bbx"
server_key: "4618fbefbd472ac41ca84105a6e08904"
server_key_iv: "d552fc974cc37b737334b557a34afa3c"
server_key_version: "encryptionkey@0"
I20260808 23:32:29.846072 10466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "3735e2e6880c4f86a96e8f160d079f07"
format_stamp: "Formatted at 2026-08-08 23:32:29 on dist-test-slave-6bbx"
server_key: "4618fbefbd472ac41ca84105a6e08904"
server_key_iv: "d552fc974cc37b737334b557a34afa3c"
server_key_version: "encryptionkey@0"
I20260808 23:32:29.850960 10466 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260808 23:32:29.854406 10488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:32:29.855581 10466 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 23:32:29.855793 10466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "3735e2e6880c4f86a96e8f160d079f07"
format_stamp: "Formatted at 2026-08-08 23:32:29 on dist-test-slave-6bbx"
server_key: "4618fbefbd472ac41ca84105a6e08904"
server_key_iv: "d552fc974cc37b737334b557a34afa3c"
server_key_version: "encryptionkey@0"
I20260808 23:32:29.855969 10466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:32:29.861366 10466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:32:29.865060 10466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:32:29.865317 10466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:32:29.882594 10466 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:39007
I20260808 23:32:29.882649 10556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:39007 every 8 connection(s)
I20260808 23:32:29.883913 10466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:32:29.887365 10557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:32:29.891584 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 10466
I20260808 23:32:29.891783 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:32:29.892211 9393 external_mini_cluster.cc:1428] Setting key 6c32d1c5976d00ee36826b2f8ccaa32e
I20260808 23:32:29.897867 10557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3735e2e6880c4f86a96e8f160d079f07: Bootstrap starting.
I20260808 23:32:29.899720 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:29.893757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49689 (local address 127.9.44.126:39007)
0808 23:32:29.894513 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:29.894524 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:29.894563 (+ 39us) server_negotiation.cc:407] Connection header received
0808 23:32:29.896579 (+ 2016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:29.896602 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:29.896912 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:29.897212 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:29.898243 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:29.898249 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:29.898262 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:32:29.898276 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:29.898344 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:32:29.898370 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:29.898530 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:29.898658 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 23:32:29.898816 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":355,"threads_started":1}
I20260808 23:32:29.901005 10557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3735e2e6880c4f86a96e8f160d079f07: Neither blocks nor log segments found. Creating new log.
I20260808 23:32:29.902030 10557 log.cc:824] T 00000000000000000000000000000000 P 3735e2e6880c4f86a96e8f160d079f07: Log is configured to *not* fsync() on all Append() calls
I20260808 23:32:29.903156 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:32:29.905052 10557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3735e2e6880c4f86a96e8f160d079f07: No bootstrap required, opened a new log
I20260808 23:32:29.910214 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:29.907982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47017 (local address 127.9.44.126:39007)
0808 23:32:29.908157 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:29.908162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:29.908597 (+ 435us) server_negotiation.cc:407] Connection header received
0808 23:32:29.908714 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:29.908719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:29.908777 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:29.908885 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:29.909647 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:29.909651 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:29.909656 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 23:32:29.909673 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:29.909719 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 23:32:29.909738 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:29.909856 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:29.909902 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 23:32:29.909949 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 23:32:29.913837 10557 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3735e2e6880c4f86a96e8f160d079f07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3735e2e6880c4f86a96e8f160d079f07" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39007 } }
I20260808 23:32:29.914134 10557 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3735e2e6880c4f86a96e8f160d079f07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 23:32:29.914203 10557 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3735e2e6880c4f86a96e8f160d079f07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3735e2e6880c4f86a96e8f160d079f07, State: Initialized, Role: FOLLOWER
I20260808 23:32:29.914688 10557 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3735e2e6880c4f86a96e8f160d079f07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3735e2e6880c4f86a96e8f160d079f07" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39007 } }
I20260808 23:32:29.914896 10557 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3735e2e6880c4f86a96e8f160d079f07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 23:32:29.914991 10557 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3735e2e6880c4f86a96e8f160d079f07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 23:32:29.915112 10557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3735e2e6880c4f86a96e8f160d079f07 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:32:29.917240 10557 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3735e2e6880c4f86a96e8f160d079f07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3735e2e6880c4f86a96e8f160d079f07" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39007 } }
I20260808 23:32:29.917761 10557 leader_election.cc:302] T 00000000000000000000000000000000 P 3735e2e6880c4f86a96e8f160d079f07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3735e2e6880c4f86a96e8f160d079f07; no voters:
I20260808 23:32:29.918262 10557 leader_election.cc:288] T 00000000000000000000000000000000 P 3735e2e6880c4f86a96e8f160d079f07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 23:32:29.918375 10569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3735e2e6880c4f86a96e8f160d079f07 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:32:29.918954 10569 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3735e2e6880c4f86a96e8f160d079f07 [term 1 LEADER]: Becoming Leader. State: Replica: 3735e2e6880c4f86a96e8f160d079f07, State: Running, Role: LEADER
I20260808 23:32:29.919464 10569 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3735e2e6880c4f86a96e8f160d079f07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3735e2e6880c4f86a96e8f160d079f07" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39007 } }
I20260808 23:32:29.920025 10557 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3735e2e6880c4f86a96e8f160d079f07 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:32:29.922137 10572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3735e2e6880c4f86a96e8f160d079f07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3735e2e6880c4f86a96e8f160d079f07. Latest consensus state: current_term: 1 leader_uuid: "3735e2e6880c4f86a96e8f160d079f07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3735e2e6880c4f86a96e8f160d079f07" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39007 } } }
I20260808 23:32:29.922528 10572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3735e2e6880c4f86a96e8f160d079f07 [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:29.922925 10571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3735e2e6880c4f86a96e8f160d079f07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3735e2e6880c4f86a96e8f160d079f07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3735e2e6880c4f86a96e8f160d079f07" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39007 } } }
I20260808 23:32:29.923221 10571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3735e2e6880c4f86a96e8f160d079f07 [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:29.925110 10575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:32:29.930070 10575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:32:29.939256 10575 catalog_manager.cc:1347] Generated new cluster ID: b0260ee0f4334ef0a31d014bfd179c2f
I20260808 23:32:29.939375 10575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:32:29.955156 10575 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:32:29.955523 10575 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:32:29.964438 10575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3735e2e6880c4f86a96e8f160d079f07: Generated new TSK 0
I20260808 23:32:29.965049 10575 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:32:29.975891 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 10466
W20260808 23:32:29.976938 10551 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49689
W20260808 23:32:29.977607 9393 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 10466: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 23:32:29.977742 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 10466
2026-08-08T23:32:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1973 ms)
[----------] 18 tests from AuthTokenIssuingTest (36577 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:32:30 dist-test-slave-6bbx krb5kdc[10598](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:32:30 dist-test-slave-6bbx krb5kdc[10598](info): set up 2 sockets
Aug 08 23:32:30 dist-test-slave-6bbx krb5kdc[10598](info): commencing operation
krb5kdc: starting...
W20260808 23:32:31.183405 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.004s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:32:31 dist-test-slave-6bbx krb5kdc[10598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:32:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:32:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:32:31.460865 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:39987
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:34513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:39987,127.9.44.125:42193,127.9.44.124:32881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:32:31.783440 10615 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 23:32:31.783775 10615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:32:31.783867 10615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:32:31.783949 10615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:32:31.791056 10615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:32:31.791147 10615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:32:31.791218 10615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:32:31.791268 10615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:32:31.791330 10615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:32:31.801133 10615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:34513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:39987,127.9.44.125:42193,127.9.44.124:32881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:39987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:32:31.802318 10615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:32:31.803542 10615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:32:31.814608 10623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:31.814788 10625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:31.814929 10622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:32:31.815644 10615 server_base.cc:1034] running on GCE node
I20260808 23:32:31.816402 10615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:32:31.817659 10615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:32:31.818881 10615 hybrid_clock.cc:648] HybridClock initialized: now 1786231951818820 us; error 37 us; skew 500 ppm
I20260808 23:32:31.819186 10615 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:32:31 dist-test-slave-6bbx krb5kdc[10598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231951, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:32:31.822230 10615 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:32:31.823559 10615 webserver.cc:460] Webserver started at http://127.9.44.126:38885/ using document root <none> and password file <none>
I20260808 23:32:31.824257 10615 fs_manager.cc:359] Metadata directory not provided
I20260808 23:32:31.824358 10615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:32:31.824651 10615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:32:31.827448 10615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "21be737f8f684a87a43e829edcae9df5"
format_stamp: "Formatted at 2026-08-08 23:32:31 on dist-test-slave-6bbx"
server_key: "ba71810efa3be0a882d819c3ac320648"
server_key_iv: "1f664ec2ff06ba153f109e9100a5532a"
server_key_version: "encryptionkey@0"
I20260808 23:32:31.828227 10615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "21be737f8f684a87a43e829edcae9df5"
format_stamp: "Formatted at 2026-08-08 23:32:31 on dist-test-slave-6bbx"
server_key: "ba71810efa3be0a882d819c3ac320648"
server_key_iv: "1f664ec2ff06ba153f109e9100a5532a"
server_key_version: "encryptionkey@0"
I20260808 23:32:31.833032 10615 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 23:32:31.836237 10634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:32:31.837543 10615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:32:31.837700 10615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "21be737f8f684a87a43e829edcae9df5"
format_stamp: "Formatted at 2026-08-08 23:32:31 on dist-test-slave-6bbx"
server_key: "ba71810efa3be0a882d819c3ac320648"
server_key_iv: "1f664ec2ff06ba153f109e9100a5532a"
server_key_version: "encryptionkey@0"
I20260808 23:32:31.837903 10615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:32:31.888612 10615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:32:31.892407 10615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:32:31.892665 10615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:32:31.909865 10615 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:39987
I20260808 23:32:31.909888 10705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:39987 every 8 connection(s)
I20260808 23:32:31.911083 10615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:32:31.914170 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 10615
I20260808 23:32:31.914358 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:32:31.914652 10706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:32:31.915014 9393 external_mini_cluster.cc:1428] Setting key 905bab24d011ca82a8f233e986182c62
I20260808 23:32:31.920941 10706 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39987 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:32:31 dist-test-slave-6bbx krb5kdc[10598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231951, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:32:31.943392 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:31.924063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32978 (local address 127.9.44.126:39987)
0808 23:32:31.924843 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:31.924856 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:31.925420 (+ 564us) server_negotiation.cc:407] Connection header received
0808 23:32:31.927413 (+ 1993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:31.927421 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:31.927722 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:31.927931 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:31.929007 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:31.930274 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:31.932131 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:31.932682 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:31.932808 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:31.936486 (+ 3678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:31.936494 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:31.936505 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:31.936537 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:31.939087 (+ 2550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:31.939849 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:31.939855 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:31.939857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:31.940004 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:31.940490 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:31.940496 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:31.940499 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:31.941106 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:31.941294 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:31.941946 (+ 652us) server_negotiation.cc:299] Negotiation successful
0808 23:32:31.942371 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":86,"server-negotiator.queue_time_us":546,"thread_start_us":331,"threads_started":1}
I20260808 23:32:31.943418 10710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:31.924063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:39987 (local address 127.0.0.1:32978)
0808 23:32:31.925241 (+ 1178us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:31.925290 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 23:32:31.926973 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:32:31.927935 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:32:31.927940 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:32:31.928474 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:32:31.928800 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:31.928818 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:31.930426 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:31.930431 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:31.931994 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:31.932014 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:31.932918 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:31.932925 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:31.933074 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:31.933703 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:32:31.933736 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:32:31.936273 (+ 2537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:32:31.939212 (+ 2939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:31.939223 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:31.939233 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:31.939732 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:31.940123 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:31.940127 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:31.940130 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:31.940324 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:31.941252 (+ 928us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:32:31.941259 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:32:31.941502 (+ 243us) client_negotiation.cc:769] Sending connection context
0808 23:32:31.941820 (+ 318us) client_negotiation.cc:240] Negotiation successful
0808 23:32:31.942519 (+ 699us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":927,"mutex_wait_us":158,"thread_start_us":711,"threads_started":1}
I20260808 23:32:31.946363 10706 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:32:31.947326 10710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:31.946904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:42193 (local address 127.0.0.1:60660)
0808 23:32:31.947084 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:31.947206 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 23:32:31.947580 10635 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42193: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.9.44.125@KRBTEST.COM; defaulting to no policy
I20260808 23:32:31.949079 10710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:31.948724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:42193 (local address 127.0.0.1:60676)
0808 23:32:31.948872 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:31.948949 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 23:32:31.949303 10706 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.125:42193: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42193: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.125.keytab.
Entry for principal kudu/127.9.44.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.125.keytab.
I20260808 23:32:31.980530 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.125:42193
--webserver_interface=127.9.44.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.125.keytab
--principal=kudu/127.9.44.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:34513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:39987,127.9.44.125:42193,127.9.44.124:32881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:32:31.998615 10706 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 } attempt: 1
I20260808 23:32:31.999724 10710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:31.999174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:42193 (local address 127.0.0.1:60688)
0808 23:32:31.999403 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:31.999546 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 23:32:32.000743 10710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.000313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:42193 (local address 127.0.0.1:60704)
0808 23:32:32.000464 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.000550 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 23:32:32.000949 10706 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.125:42193: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42193: connect: Connection refused (error 111)
I20260808 23:32:32.069221 10706 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 } attempt: 2
I20260808 23:32:32.070354 10710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.069906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:42193 (local address 127.0.0.1:60708)
0808 23:32:32.070112 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.070209 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 23:32:32.071398 10710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.071026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:42193 (local address 127.0.0.1:60718)
0808 23:32:32.071177 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.071266 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 23:32:32.071586 10706 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.125:42193: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42193: connect: Connection refused (error 111)
I20260808 23:32:32.162850 10706 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 } attempt: 3
I20260808 23:32:32.164054 10710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.163583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:42193 (local address 127.0.0.1:60732)
0808 23:32:32.163813 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.163905 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260808 23:32:32.165127 10710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.164765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:42193 (local address 127.0.0.1:60742)
0808 23:32:32.164916 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.164987 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 23:32:32.165315 10706 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.125:42193: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42193: connect: Connection refused (error 111)
W20260808 23:32:32.305393 10715 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 23:32:32.305706 10715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:32:32.305800 10715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:32:32.305917 10715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 23:32:32.308522 10706 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 } attempt: 4
I20260808 23:32:32.309583 10710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.309097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:42193 (local address 127.0.0.1:60748)
0808 23:32:32.309374 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.309458 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260808 23:32:32.310567 10710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.310234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:42193 (local address 127.0.0.1:60758)
0808 23:32:32.310372 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.310448 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 23:32:32.310735 10706 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.125:42193: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42193: connect: Connection refused (error 111)
W20260808 23:32:32.313094 10715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:32:32.313182 10715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:32:32.313271 10715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:32:32.313333 10715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:32:32.313414 10715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:32:32.322656 10715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:34513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:39987,127.9.44.125:42193,127.9.44.124:32881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.125.keytab
--principal=kudu/127.9.44.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.125:42193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.44.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:32:32.323799 10715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:32:32.324996 10715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:32:32.335856 10724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:32.336077 10720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:32.335909 10721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:32:32.336752 10715 server_base.cc:1034] running on GCE node
I20260808 23:32:32.337446 10715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:32:32.338620 10715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:32:32.339808 10715 hybrid_clock.cc:648] HybridClock initialized: now 1786231952339733 us; error 46 us; skew 500 ppm
I20260808 23:32:32.340073 10715 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:32:32 dist-test-slave-6bbx krb5kdc[10598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231952, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:32:32.343148 10715 init.cc:377] Logged in from keytab as kudu/127.9.44.125@KRBTEST.COM (short username kudu)
I20260808 23:32:32.344426 10715 webserver.cc:460] Webserver started at http://127.9.44.125:42651/ using document root <none> and password file <none>
I20260808 23:32:32.345021 10715 fs_manager.cc:359] Metadata directory not provided
I20260808 23:32:32.345103 10715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:32:32.345358 10715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:32:32.348129 10715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-1/data/instance:
uuid: "207112f995d144c389b884dcb9f6ac83"
format_stamp: "Formatted at 2026-08-08 23:32:32 on dist-test-slave-6bbx"
server_key: "e5b536bf53cc68cafb1623363cd2525b"
server_key_iv: "51bd4ec2a28cff432befdad9f2c86eb8"
server_key_version: "encryptionkey@0"
I20260808 23:32:32.348905 10715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-1/wal/instance:
uuid: "207112f995d144c389b884dcb9f6ac83"
format_stamp: "Formatted at 2026-08-08 23:32:32 on dist-test-slave-6bbx"
server_key: "e5b536bf53cc68cafb1623363cd2525b"
server_key_iv: "51bd4ec2a28cff432befdad9f2c86eb8"
server_key_version: "encryptionkey@0"
I20260808 23:32:32.353505 10715 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260808 23:32:32.356581 10736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:32:32.357752 10715 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 23:32:32.357936 10715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-1/wal
uuid: "207112f995d144c389b884dcb9f6ac83"
format_stamp: "Formatted at 2026-08-08 23:32:32 on dist-test-slave-6bbx"
server_key: "e5b536bf53cc68cafb1623363cd2525b"
server_key_iv: "51bd4ec2a28cff432befdad9f2c86eb8"
server_key_version: "encryptionkey@0"
I20260808 23:32:32.358127 10715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:32:32.383277 10715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:32:32.386756 10715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:32:32.386965 10715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:32:32.403234 10715 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.125:42193
I20260808 23:32:32.403208 10806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.125:42193 every 8 connection(s)
I20260808 23:32:32.404428 10715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-1/data/info.pb
I20260808 23:32:32.407932 10807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:32:32.413475 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 10715
I20260808 23:32:32.413630 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-1/wal/instance
I20260808 23:32:32.414041 9393 external_mini_cluster.cc:1428] Setting key cf9f1c9579e642e0d13c091c16f87871
I20260808 23:32:32.414103 10807 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39987 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:32:32 dist-test-slave-6bbx krb5kdc[10598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231952, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.125@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:32:32.435092 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.416631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32994 (local address 127.9.44.126:39987)
0808 23:32:32.416829 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:32.416834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:32.417893 (+ 1059us) server_negotiation.cc:407] Connection header received
0808 23:32:32.420060 (+ 2167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:32.420065 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:32.420132 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:32.420218 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:32.421974 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.422843 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.424810 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.425318 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.425428 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:32.429159 (+ 3731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:32.429165 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:32.429170 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:32.429216 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:32.432313 (+ 3097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:32.432973 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:32.432979 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:32.432981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:32.433123 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:32.433499 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:32.433503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:32.433506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:32.433831 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:32.433976 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:32.434579 (+ 603us) server_negotiation.cc:299] Negotiation successful
0808 23:32:32.434802 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 23:32:32.435724 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.416984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:39987 (local address 127.0.0.1:32994)
0808 23:32:32.417715 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.417759 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 23:32:32.419804 (+ 2045us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:32:32.420451 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:32:32.420461 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:32:32.421100 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:32:32.421807 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:32.421832 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.423005 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.423010 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:32.424675 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:32.424690 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.425534 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.425541 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:32.425801 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:32.426470 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:32:32.426498 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:32:32.428937 (+ 2439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:32:32.432437 (+ 3500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:32.432443 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:32.432453 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:32.432847 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:32.433214 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:32.433217 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:32.433219 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:32.433382 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:32.433934 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:32:32.433939 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:32:32.434203 (+ 264us) client_negotiation.cc:769] Sending connection context
0808 23:32:32.434413 (+ 210us) client_negotiation.cc:240] Negotiation successful
0808 23:32:32.434739 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":291,"threads_started":1}
I20260808 23:32:32.437310 10807 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:32:32 dist-test-slave-6bbx krb5kdc[10598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231952, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.125@KRBTEST.COM for kudu/127.9.44.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.44.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.124@KRBTEST.COM" created.
I20260808 23:32:32.450824 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.438005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:42193 (local address 127.0.0.1:60772)
0808 23:32:32.438179 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.438196 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:32:32.438268 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:32:32.439200 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:32:32.439204 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:32:32.439579 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:32:32.439739 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:32.439748 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.441069 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.441075 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:32.441945 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:32.441958 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.442805 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.442811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:32.442988 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:32.443502 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:32:32.443517 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:32:32.445551 (+ 2034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:32:32.448508 (+ 2957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:32.448512 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:32.448514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:32.449026 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:32.449409 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:32.449413 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:32.449415 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:32.449545 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:32.450186 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:32:32.450189 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:32:32.450297 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 23:32:32.450398 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 23:32:32.450570 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 23:32:32.451182 10812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.438217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60772 (local address 127.9.44.125:42193)
0808 23:32:32.438834 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:32.438839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:32.438871 (+ 32us) server_negotiation.cc:407] Connection header received
0808 23:32:32.438914 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:32.438919 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:32.439057 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:32.439187 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:32.439897 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.440941 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.442072 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.442677 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.442785 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:32.445738 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:32.445748 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:32.445759 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:32.445786 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:32.448397 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:32.449157 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:32.449165 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:32.449169 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:32.449315 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:32.449645 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:32.449650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:32.449653 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:32.450061 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:32.450269 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:32.450559 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 23:32:32.450797 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":365,"threads_started":1}
I20260808 23:32:32.453274 10807 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:32:32.454334 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.453928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:32881 (local address 127.0.0.1:36206)
0808 23:32:32.454087 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.454211 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 23:32:32.454557 10739 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
I20260808 23:32:32.455994 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.455701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:32881 (local address 127.0.0.1:36214)
0808 23:32:32.455831 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.455901 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 23:32:32.456183 10807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:32881: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.124.keytab.
Entry for principal kudu/127.9.44.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.124.keytab.
I20260808 23:32:32.479789 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.124:32881
--webserver_interface=127.9.44.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.124.keytab
--principal=kudu/127.9.44.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:34513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:39987,127.9.44.125:42193,127.9.44.124:32881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:32:32.505479 10807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } attempt: 1
I20260808 23:32:32.506693 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.506172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:32881 (local address 127.0.0.1:36216)
0808 23:32:32.506389 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.506510 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 23:32:32.507695 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.507304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:32881 (local address 127.0.0.1:36232)
0808 23:32:32.507439 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.507542 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 23:32:32.507910 10807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:32881: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
I20260808 23:32:32.576130 10807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } attempt: 2
I20260808 23:32:32.577263 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.576749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:32881 (local address 127.0.0.1:36244)
0808 23:32:32.576965 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.577077 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 23:32:32.578285 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.577891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:32881 (local address 127.0.0.1:36258)
0808 23:32:32.578042 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.578133 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 23:32:32.578504 10807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:32881: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
I20260808 23:32:32.609987 10706 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 } attempt: 5
Aug 08 23:32:32 dist-test-slave-6bbx krb5kdc[10598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231951, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for kudu/127.9.44.125@KRBTEST.COM
I20260808 23:32:32.623925 10812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.610789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60780 (local address 127.9.44.125:42193)
0808 23:32:32.610969 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:32.610974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:32.610993 (+ 19us) server_negotiation.cc:407] Connection header received
0808 23:32:32.611193 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:32.611197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:32.611271 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:32.611354 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:32.612384 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.613081 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.614779 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.615193 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.615281 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:32.618892 (+ 3611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:32.618898 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:32.618901 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:32.618933 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:32.621457 (+ 2524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:32.622227 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:32.622231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:32.622234 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:32.622352 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:32.622713 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:32.622716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:32.622718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:32.622983 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:32.623090 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:32.623490 (+ 400us) server_negotiation.cc:299] Negotiation successful
0808 23:32:32.623646 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 23:32:32.624085 10710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.610691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:42193 (local address 127.0.0.1:60780)
0808 23:32:32.610896 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.610919 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 23:32:32.611067 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:32:32.611431 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:32:32.611437 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:32:32.611991 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:32:32.612227 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:32.612242 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.613219 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.613224 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:32.614615 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:32.614630 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.615335 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.615341 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:32.615558 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:32.616197 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:32:32.616221 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:32:32.618685 (+ 2464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:32:32.621584 (+ 2899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:32.621589 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:32.621593 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:32.622108 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:32.622447 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:32.622452 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:32.622455 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:32.622623 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:32.623093 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:32:32.623097 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:32:32.623273 (+ 176us) client_negotiation.cc:769] Sending connection context
0808 23:32:32.623425 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 23:32:32.623753 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 23:32:32.625260 10706 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:32:32.626345 10710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.625898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:32881 (local address 127.0.0.1:36260)
0808 23:32:32.626077 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.626171 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 23:32:32.627317 10710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.626944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:32881 (local address 127.0.0.1:36272)
0808 23:32:32.627117 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.627201 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 23:32:32.627525 10706 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:32881: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
I20260808 23:32:32.669734 10807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } attempt: 3
I20260808 23:32:32.670773 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.670261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:32881 (local address 127.0.0.1:36282)
0808 23:32:32.670471 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.670578 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 23:32:32.671887 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.671349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:32881 (local address 127.0.0.1:36292)
0808 23:32:32.671598 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.671734 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
W20260808 23:32:32.672104 10807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:32881: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
I20260808 23:32:32.678766 10706 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } attempt: 1
I20260808 23:32:32.679767 10710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.679272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:32881 (local address 127.0.0.1:36296)
0808 23:32:32.679458 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.679553 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 23:32:32.680675 10710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.680325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:32881 (local address 127.0.0.1:36304)
0808 23:32:32.680475 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.680556 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 23:32:32.680912 10706 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:32881: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
I20260808 23:32:32.749153 10706 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } attempt: 2
I20260808 23:32:32.750342 10710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.749785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:32881 (local address 127.0.0.1:36308)
0808 23:32:32.750039 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.750149 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 23:32:32.751317 10710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.750957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:32881 (local address 127.0.0.1:36316)
0808 23:32:32.751096 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.751182 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 23:32:32.751520 10706 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:32881: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
W20260808 23:32:32.808568 10816 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 23:32:32.808843 10816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:32:32.808912 10816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:32:32.808987 10816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 23:32:32.815301 10807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } attempt: 4
W20260808 23:32:32.815853 10816 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:32:32.815927 10816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:32:32.815999 10816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:32:32.816068 10816 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:32:32.816154 10816 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:32:32.816453 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.815899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:32881 (local address 127.0.0.1:36324)
0808 23:32:32.816166 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.816274 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260808 23:32:32.817431 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.817073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:32881 (local address 127.0.0.1:36338)
0808 23:32:32.817220 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.817303 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 23:32:32.817641 10807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:32881: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
I20260808 23:32:32.825996 10816 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:34513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:39987,127.9.44.125:42193,127.9.44.124:32881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.124.keytab
--principal=kudu/127.9.44.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.124:32881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.44.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:32:32.827085 10816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:32:32.828342 10816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:32:32.839028 10825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:32.838985 10828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:32.839061 10826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:32:32.839939 10816 server_base.cc:1034] running on GCE node
I20260808 23:32:32.840616 10816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:32:32.841905 10816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:32:32.843122 10816 hybrid_clock.cc:648] HybridClock initialized: now 1786231952843030 us; error 50 us; skew 500 ppm
I20260808 23:32:32.843425 10816 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:32:32 dist-test-slave-6bbx krb5kdc[10598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231952, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:32:32.846304 10816 init.cc:377] Logged in from keytab as kudu/127.9.44.124@KRBTEST.COM (short username kudu)
I20260808 23:32:32.847565 10816 webserver.cc:460] Webserver started at http://127.9.44.124:38701/ using document root <none> and password file <none>
I20260808 23:32:32.848215 10816 fs_manager.cc:359] Metadata directory not provided
I20260808 23:32:32.848320 10816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:32:32.848608 10816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:32:32.851343 10816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-2/data/instance:
uuid: "8b3137677e8d496ab55d11a56b24f6bd"
format_stamp: "Formatted at 2026-08-08 23:32:32 on dist-test-slave-6bbx"
server_key: "64f0ff69c99df1c434423cc5923d2a4b"
server_key_iv: "5bdb843aee3bb4bcb0e3604633a5437a"
server_key_version: "encryptionkey@0"
I20260808 23:32:32.852100 10816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-2/wal/instance:
uuid: "8b3137677e8d496ab55d11a56b24f6bd"
format_stamp: "Formatted at 2026-08-08 23:32:32 on dist-test-slave-6bbx"
server_key: "64f0ff69c99df1c434423cc5923d2a4b"
server_key_iv: "5bdb843aee3bb4bcb0e3604633a5437a"
server_key_version: "encryptionkey@0"
I20260808 23:32:32.856791 10816 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 23:32:32.857710 10706 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } attempt: 3
I20260808 23:32:32.858595 10710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.858182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:32881 (local address 127.0.0.1:36340)
0808 23:32:32.858367 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.858444 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 23:32:32.859594 10710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.859192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:32881 (local address 127.0.0.1:36346)
0808 23:32:32.859406 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.859480 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260808 23:32:32.859882 10706 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:32881: Network error: Client connection negotiation failed: client connection to 127.9.44.124:32881: connect: Connection refused (error 111)
I20260808 23:32:32.860173 10839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:32:32.861467 10816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 23:32:32.861639 10816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-2/wal
uuid: "8b3137677e8d496ab55d11a56b24f6bd"
format_stamp: "Formatted at 2026-08-08 23:32:32 on dist-test-slave-6bbx"
server_key: "64f0ff69c99df1c434423cc5923d2a4b"
server_key_iv: "5bdb843aee3bb4bcb0e3604633a5437a"
server_key_version: "encryptionkey@0"
I20260808 23:32:32.861847 10816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:32:32.878300 10816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:32:32.881847 10816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:32:32.882076 10816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:32:32.898932 10816 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.124:32881
I20260808 23:32:32.898962 10917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.124:32881 every 8 connection(s)
I20260808 23:32:32.900131 10816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-2/data/info.pb
I20260808 23:32:32.903611 10919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:32:32.904796 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 10816
I20260808 23:32:32.904999 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786231597224934-9393-0/minicluster-data/master-2/wal/instance
I20260808 23:32:32.905589 9393 external_mini_cluster.cc:1428] Setting key 4edad543e3b7dbee1e6816efb8170061
I20260808 23:32:32.913287 10919 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39987 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:32:32 dist-test-slave-6bbx krb5kdc[10598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.124@KRBTEST.COM
Aug 08 23:32:32 dist-test-slave-6bbx krb5kdc[10598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231952, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.124@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:32:32.930194 10923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.907417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36356 (local address 127.9.44.124:32881)
0808 23:32:32.908179 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:32.908189 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:32.908235 (+ 46us) server_negotiation.cc:407] Connection header received
0808 23:32:32.909964 (+ 1729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:32.909986 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:32.910337 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:32.910675 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:32.912177 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.913408 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.914705 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.915347 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.915511 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:32.923434 (+ 7923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:32.923442 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:32.923453 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:32.923486 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:32.926382 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:32.927104 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:32.927109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:32.927114 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:32.927254 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:32.927648 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:32.927653 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:32.927656 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:32.928348 (+ 692us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:32.928557 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:32.928988 (+ 431us) server_negotiation.cc:299] Negotiation successful
0808 23:32:32.929286 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":356,"threads_started":1}
I20260808 23:32:32.930378 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.916161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33000 (local address 127.9.44.126:39987)
0808 23:32:32.916317 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:32.916321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:32.917246 (+ 925us) server_negotiation.cc:407] Connection header received
0808 23:32:32.917427 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:32.917431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:32.917489 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:32.917568 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:32.918601 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.919288 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.921030 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.921447 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.921546 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:32.924782 (+ 3236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:32.924787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:32.924790 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:32.924821 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:32.927523 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:32.928333 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:32.928337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:32.928339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:32.928505 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:32.928904 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:32.928909 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:32.928912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:32.929251 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:32.929393 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:32.929829 (+ 436us) server_negotiation.cc:299] Negotiation successful
0808 23:32:32.930042 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 23:32:32.930459 10924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.916437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:39987 (local address 127.0.0.1:33000)
0808 23:32:32.917110 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.917148 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 23:32:32.917312 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:32:32.917640 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:32:32.917646 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:32:32.918214 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:32:32.918461 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:32.918469 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.919431 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.919435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:32.920884 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:32.920902 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.921661 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.921669 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:32.921852 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:32.922452 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:32:32.922473 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:32:32.924603 (+ 2130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:32:32.927651 (+ 3048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:32.927683 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:32.927690 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:32.928218 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:32.928620 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:32.928623 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:32.928626 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:32.928762 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:32.929392 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:32:32.929399 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:32:32.929546 (+ 147us) client_negotiation.cc:769] Sending connection context
0808 23:32:32.929688 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 23:32:32.929991 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":364,"threads_started":1}
I20260808 23:32:32.931975 10919 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:32:32 dist-test-slave-6bbx krb5kdc[10598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
Aug 08 23:32:32 dist-test-slave-6bbx krb5kdc[10598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231952, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.124@KRBTEST.COM for kudu/127.9.44.125@KRBTEST.COM
I20260808 23:32:32.953086 10812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.933008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60786 (local address 127.9.44.125:42193)
0808 23:32:32.935374 (+ 2366us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:32.935381 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:32.935403 (+ 22us) server_negotiation.cc:407] Connection header received
0808 23:32:32.935462 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:32.935465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:32.935529 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:32.935624 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:32.940546 (+ 4922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.941390 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.942697 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.943256 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.943375 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:32.947275 (+ 3900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:32.947282 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:32.947287 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:32.947322 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:32.950447 (+ 3125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:32.951053 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:32.951058 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:32.951061 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:32.951212 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:32.951569 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:32.951574 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:32.951576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:32.951868 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:32.951978 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:32.952490 (+ 512us) server_negotiation.cc:299] Negotiation successful
0808 23:32:32.952713 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 23:32:32.953946 10924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.932961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:42193 (local address 127.0.0.1:60786)
0808 23:32:32.933086 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.933103 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 23:32:32.933199 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:32:32.939912 (+ 6713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:32:32.939917 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:32:32.940232 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:32:32.940380 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:32.940391 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.941518 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.941523 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:32.942572 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:32.942586 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.943402 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.943408 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:32.943586 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:32.944610 (+ 1024us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:32:32.944632 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:32:32.947067 (+ 2435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:32:32.950576 (+ 3509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:32.950582 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:32.950586 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:32.950938 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:32.951333 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:32.951337 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:32.951339 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:32.951474 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:32.952059 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:32:32.952063 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:32:32.952215 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 23:32:32.953364 (+ 1149us) client_negotiation.cc:240] Negotiation successful
0808 23:32:32.953630 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 23:32:32.955016 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.934050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33014 (local address 127.9.44.126:39987)
0808 23:32:32.934190 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:32.934195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:32.934438 (+ 243us) server_negotiation.cc:407] Connection header received
0808 23:32:32.934491 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:32.934496 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:32.934560 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:32.934647 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:32.936059 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.937119 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.939915 (+ 2796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.940312 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.940406 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:32.945865 (+ 5459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:32.945869 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:32.945873 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:32.945902 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:32.952152 (+ 6250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:32.952805 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:32.952811 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:32.952815 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:32.952963 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:32.953427 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:32.953432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:32.953436 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:32.953767 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:32.954046 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:32.954528 (+ 482us) server_negotiation.cc:299] Negotiation successful
0808 23:32:32.954726 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 23:32:32.955097 10919 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:32:32 dist-test-slave-6bbx krb5kdc[10598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231952, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.124@KRBTEST.COM for kudu/127.9.44.124@KRBTEST.COM
Aug 08 23:32:32 dist-test-slave-6bbx krb5kdc[10598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.125@KRBTEST.COM
I20260808 23:32:32.969666 10924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.955894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:32881 (local address 127.0.0.1:36372)
0808 23:32:32.956064 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:32.956080 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:32:32.956157 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:32:32.956515 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:32:32.956520 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:32:32.956963 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:32:32.957126 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:32.957136 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.958433 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.958437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:32.959713 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:32.959729 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.960666 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.960673 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:32.960902 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:32.961481 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:32:32.961510 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:32:32.963769 (+ 2259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:32:32.967029 (+ 3260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:32.967034 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:32.967038 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:32.967481 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:32.967917 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:32.967921 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:32.967924 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:32.968073 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:32.968692 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:32:32.968696 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:32:32.968854 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 23:32:32.968999 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 23:32:32.969274 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 23:32:32.969723 10923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.956097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36372 (local address 127.9.44.124:32881)
0808 23:32:32.956241 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:32.956246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:32.956262 (+ 16us) server_negotiation.cc:407] Connection header received
0808 23:32:32.956308 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:32.956312 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:32.956386 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:32.956472 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:32.957252 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.958266 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.959900 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.960498 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.960633 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:32.963949 (+ 3316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:32.963953 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:32.963956 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:32.963982 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:32.966910 (+ 2928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:32.967602 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:32.967606 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:32.967610 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:32.967776 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:32.968187 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:32.968191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:32.968194 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:32.968518 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:32.968647 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:32.969095 (+ 448us) server_negotiation.cc:299] Negotiation successful
0808 23:32:32.969427 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":115,"server-negotiator.queue_time_us":50}
I20260808 23:32:32.973352 10812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:32.956653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60802 (local address 127.9.44.125:42193)
0808 23:32:32.956801 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:32.956807 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:32.956850 (+ 43us) server_negotiation.cc:407] Connection header received
0808 23:32:32.957048 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:32.957053 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:32.957120 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:32.957184 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:32.958748 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.959800 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.961551 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:32.962044 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:32.962114 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:32.967249 (+ 5135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:32.967255 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:32.967260 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:32.967307 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:32.970641 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:32.971383 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:32.971389 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:32.971392 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:32.971550 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:32.971953 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:32.971958 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:32.971962 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:32.972285 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:32.972555 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:32.972819 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 23:32:32.973031 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 23:32:32.976128 10919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b3137677e8d496ab55d11a56b24f6bd: Bootstrap starting.
I20260808 23:32:32.980064 10919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b3137677e8d496ab55d11a56b24f6bd: Neither blocks nor log segments found. Creating new log.
I20260808 23:32:32.981302 10919 log.cc:824] T 00000000000000000000000000000000 P 8b3137677e8d496ab55d11a56b24f6bd: Log is configured to *not* fsync() on all Append() calls
I20260808 23:32:32.986209 10919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b3137677e8d496ab55d11a56b24f6bd: No bootstrap required, opened a new log
I20260808 23:32:32.986634 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:32:32.990201 10672 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "21be737f8f684a87a43e829edcae9df5"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33014
I20260808 23:32:32.996484 10919 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b3137677e8d496ab55d11a56b24f6bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21be737f8f684a87a43e829edcae9df5" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39987 } } peers { permanent_uuid: "207112f995d144c389b884dcb9f6ac83" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 } } peers { permanent_uuid: "8b3137677e8d496ab55d11a56b24f6bd" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } }
I20260808 23:32:32.996937 10919 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b3137677e8d496ab55d11a56b24f6bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b3137677e8d496ab55d11a56b24f6bd, State: Initialized, Role: FOLLOWER
I20260808 23:32:32.997416 10919 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b3137677e8d496ab55d11a56b24f6bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21be737f8f684a87a43e829edcae9df5" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39987 } } peers { permanent_uuid: "207112f995d144c389b884dcb9f6ac83" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 } } peers { permanent_uuid: "8b3137677e8d496ab55d11a56b24f6bd" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } }
I20260808 23:32:32.998286 10927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b3137677e8d496ab55d11a56b24f6bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21be737f8f684a87a43e829edcae9df5" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39987 } } peers { permanent_uuid: "207112f995d144c389b884dcb9f6ac83" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 } } peers { permanent_uuid: "8b3137677e8d496ab55d11a56b24f6bd" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } } }
I20260808 23:32:32.998588 10927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b3137677e8d496ab55d11a56b24f6bd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:32:32.999311 10919 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b3137677e8d496ab55d11a56b24f6bd [sys.catalog]: configured and running, proceeding with master startup.
W20260808 23:32:33.008198 10941 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8b3137677e8d496ab55d11a56b24f6bd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:32:33.008325 10941 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:32:33.005820 10816 master_runner.cc:186] Error getting master registration for 127.9.44.126:39987: OK, instance_id { permanent_uuid: "21be737f8f684a87a43e829edcae9df5" instance_seqno: 1786231951905003 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 23:32:33.016856 10816 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 23:32:33.037106 10706 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } attempt: 4
Aug 08 23:32:33 dist-test-slave-6bbx krb5kdc[10598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231951, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for kudu/127.9.44.124@KRBTEST.COM
I20260808 23:32:33.053629 10710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:33.037738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:32881 (local address 127.0.0.1:36384)
0808 23:32:33.037942 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:33.037962 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 23:32:33.038095 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:32:33.038436 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:32:33.038441 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:32:33.038960 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:32:33.039200 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:33.039214 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:33.040687 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:33.040692 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:33.042228 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:33.042244 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:33.043274 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:33.043279 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:33.043486 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:33.044179 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:32:33.044201 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:32:33.046899 (+ 2698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:32:33.050813 (+ 3914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:33.050819 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:33.050823 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:33.051339 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:33.051768 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:33.051773 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:33.051776 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:33.051955 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:33.052630 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:32:33.052635 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:32:33.052799 (+ 164us) client_negotiation.cc:769] Sending connection context
0808 23:32:33.052947 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 23:32:33.053240 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 23:32:33.053643 10923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:33.037772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36384 (local address 127.9.44.124:32881)
0808 23:32:33.037965 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:33.037970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:33.038049 (+ 79us) server_negotiation.cc:407] Connection header received
0808 23:32:33.038216 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:33.038221 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:33.038301 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:33.038416 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:33.039366 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:33.040536 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:33.042375 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:33.043142 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:33.043270 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:33.047071 (+ 3801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:33.047078 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:33.047083 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:33.047128 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:33.050685 (+ 3557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:33.051456 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:33.051462 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:33.051466 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:33.051607 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:33.052067 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:33.052072 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:33.052076 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:33.052503 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:33.052665 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:33.053075 (+ 410us) server_negotiation.cc:299] Negotiation successful
0808 23:32:33.053290 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 23:32:33.059602 10706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5: Bootstrap starting.
I20260808 23:32:33.063573 10706 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5: Neither blocks nor log segments found. Creating new log.
I20260808 23:32:33.064879 10706 log.cc:824] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5: Log is configured to *not* fsync() on all Append() calls
I20260808 23:32:33.067806 10706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5: No bootstrap required, opened a new log
I20260808 23:32:33.072983 10706 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21be737f8f684a87a43e829edcae9df5" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39987 } } peers { permanent_uuid: "207112f995d144c389b884dcb9f6ac83" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 } } peers { permanent_uuid: "8b3137677e8d496ab55d11a56b24f6bd" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } }
I20260808 23:32:33.073288 10706 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21be737f8f684a87a43e829edcae9df5, State: Initialized, Role: FOLLOWER
I20260808 23:32:33.073693 10706 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21be737f8f684a87a43e829edcae9df5" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39987 } } peers { permanent_uuid: "207112f995d144c389b884dcb9f6ac83" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 } } peers { permanent_uuid: "8b3137677e8d496ab55d11a56b24f6bd" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } }
I20260808 23:32:33.074625 10944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21be737f8f684a87a43e829edcae9df5" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39987 } } peers { permanent_uuid: "207112f995d144c389b884dcb9f6ac83" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 } } peers { permanent_uuid: "8b3137677e8d496ab55d11a56b24f6bd" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } } }
I20260808 23:32:33.075038 10944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:32:33.075841 10706 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:32:33.078697 10672 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 23:32:33.078908 10672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:32:33.082165 10672 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21be737f8f684a87a43e829edcae9df5" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39987 } } peers { permanent_uuid: "207112f995d144c389b884dcb9f6ac83" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 } } peers { permanent_uuid: "8b3137677e8d496ab55d11a56b24f6bd" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } }
I20260808 23:32:33.084555 10881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "21be737f8f684a87a43e829edcae9df5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8b3137677e8d496ab55d11a56b24f6bd"
I20260808 23:32:33.085043 10881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b3137677e8d496ab55d11a56b24f6bd [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:32:33.088197 10672 leader_election.cc:288] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5 [CANDIDATE]: Term 1 election: Requested vote from peers 207112f995d144c389b884dcb9f6ac83 (127.9.44.125:42193), 8b3137677e8d496ab55d11a56b24f6bd (127.9.44.124:32881)
I20260808 23:32:33.088495 10881 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8b3137677e8d496ab55d11a56b24f6bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21be737f8f684a87a43e829edcae9df5 in term 1.
I20260808 23:32:33.089140 10636 leader_election.cc:302] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21be737f8f684a87a43e829edcae9df5, 8b3137677e8d496ab55d11a56b24f6bd; no voters:
I20260808 23:32:33.089563 10944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:32:33.090153 10944 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5 [term 1 LEADER]: Becoming Leader. State: Replica: 21be737f8f684a87a43e829edcae9df5, State: Running, Role: LEADER
I20260808 23:32:33.090715 10944 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21be737f8f684a87a43e829edcae9df5" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39987 } } peers { permanent_uuid: "207112f995d144c389b884dcb9f6ac83" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 } } peers { permanent_uuid: "8b3137677e8d496ab55d11a56b24f6bd" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } }
I20260808 23:32:33.094043 10948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21be737f8f684a87a43e829edcae9df5. Latest consensus state: current_term: 1 leader_uuid: "21be737f8f684a87a43e829edcae9df5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21be737f8f684a87a43e829edcae9df5" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39987 } } peers { permanent_uuid: "207112f995d144c389b884dcb9f6ac83" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 } } peers { permanent_uuid: "8b3137677e8d496ab55d11a56b24f6bd" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } } }
I20260808 23:32:33.094414 10948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5 [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:33.099987 10957 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:32:33.116832 10807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } attempt: 5
I20260808 23:32:33.117758 10812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:33.093801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60812 (local address 127.9.44.125:42193)
0808 23:32:33.094003 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:33.094009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:33.094088 (+ 79us) server_negotiation.cc:407] Connection header received
0808 23:32:33.094383 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:33.094388 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:33.094473 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:33.094551 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:33.098596 (+ 4045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:33.099754 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:33.102932 (+ 3178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:33.103605 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:33.103719 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:33.109220 (+ 5501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:33.109227 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:33.109232 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:33.109294 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:33.113015 (+ 3721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:33.114386 (+ 1371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:33.114404 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:33.114408 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:33.114572 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:33.115305 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:33.115312 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:33.115315 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:33.115735 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:33.115963 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:33.117052 (+ 1089us) server_negotiation.cc:299] Negotiation successful
0808 23:32:33.117323 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 23:32:33.132799 10923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:33.095494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36390 (local address 127.9.44.124:32881)
0808 23:32:33.095699 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:33.095704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:33.109125 (+ 13421us) server_negotiation.cc:407] Connection header received
0808 23:32:33.109564 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:33.109570 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:33.109648 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:33.109719 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:33.111605 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:33.112699 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:33.117750 (+ 5051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:33.118471 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:33.118573 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:33.125273 (+ 6700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:33.125279 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:33.125284 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:33.125324 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:33.128786 (+ 3462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:33.129646 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:33.129652 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:33.129656 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:33.129826 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:33.130268 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:33.130294 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:33.130297 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:33.130622 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:33.130850 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:33.131566 (+ 716us) server_negotiation.cc:299] Negotiation successful
0808 23:32:33.132056 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 23:32:33.134583 10848 tablet.cc:2369] T 00000000000000000000000000000000 P 8b3137677e8d496ab55d11a56b24f6bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:32:33.136636 10957 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:32:33.139828 10709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:33.094790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33016 (local address 127.9.44.126:39987)
0808 23:32:33.094951 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:33.094957 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:33.115410 (+ 20453us) server_negotiation.cc:407] Connection header received
0808 23:32:33.115636 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:33.115641 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:33.115740 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:33.115823 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:33.119810 (+ 3987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:33.120968 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:33.122978 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:33.123815 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:33.123940 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:33.129141 (+ 5201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:33.129149 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:33.129154 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:33.129203 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:33.135826 (+ 6623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:33.137026 (+ 1200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:33.137031 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:33.137034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:33.137268 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:33.137851 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:33.137855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:33.137858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:33.138203 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:33.138439 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:33.139162 (+ 723us) server_negotiation.cc:299] Negotiation successful
0808 23:32:33.139403 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 23:32:33.145354 10640 tablet.cc:2369] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 23:32:33 dist-test-slave-6bbx krb5kdc[10598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231952, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.125@KRBTEST.COM for kudu/127.9.44.124@KRBTEST.COM
I20260808 23:32:33.158310 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:33.117367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:32881 (local address 127.0.0.1:36406)
0808 23:32:33.117712 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:33.117730 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:32:33.117819 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:32:33.130930 (+ 13111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:32:33.130934 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:32:33.131416 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:32:33.131612 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:33.131627 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:33.140145 (+ 8518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:33.140151 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:33.141302 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:33.141316 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:33.142249 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:33.142255 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:33.142454 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:33.143432 (+ 978us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:32:33.143453 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:32:33.147340 (+ 3887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:32:33.153939 (+ 6599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:33.153945 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:33.153971 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:33.154746 (+ 775us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:33.155452 (+ 706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:33.155457 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:33.155482 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:33.155755 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:33.156795 (+ 1040us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:32:33.156800 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:32:33.157031 (+ 231us) client_negotiation.cc:769] Sending connection context
0808 23:32:33.157243 (+ 212us) client_negotiation.cc:240] Negotiation successful
0808 23:32:33.157684 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240}
I20260808 23:32:33.160238 10968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:33.118117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36406 (local address 127.9.44.124:32881)
0808 23:32:33.130361 (+ 12244us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:33.130366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:33.130388 (+ 22us) server_negotiation.cc:407] Connection header received
0808 23:32:33.130429 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:33.130433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:33.130492 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:33.131794 (+ 1302us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:33.132085 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:33.133040 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:33.141474 (+ 8434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:33.142076 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:33.142203 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:33.147626 (+ 5423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:33.147635 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:33.147641 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:33.147718 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:33.153710 (+ 5992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:33.154980 (+ 1270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:33.154984 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:33.154987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:33.155249 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:33.155973 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:33.155978 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:33.155981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:33.156619 (+ 638us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:33.158894 (+ 2275us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:33.159150 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 23:32:33.159556 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12102,"thread_start_us":12440,"threads_started":1}
I20260808 23:32:33.166231 10807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 207112f995d144c389b884dcb9f6ac83: Bootstrap starting.
I20260808 23:32:33.168942 10881 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8b3137677e8d496ab55d11a56b24f6bd [term 1 FOLLOWER]: Refusing update from remote peer 21be737f8f684a87a43e829edcae9df5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:32:33.169951 10948 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b3137677e8d496ab55d11a56b24f6bd" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:32:33.163944 10977 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 23:32:33.175550 10615 master_runner.cc:186] Error getting master registration for 127.9.44.125:42193: OK, instance_id { permanent_uuid: "207112f995d144c389b884dcb9f6ac83" instance_seqno: 1786231952398459 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 23:32:33.185711 10615 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 23:32:33.187502 10807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 207112f995d144c389b884dcb9f6ac83: Neither blocks nor log segments found. Creating new log.
I20260808 23:32:33.191130 10927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b3137677e8d496ab55d11a56b24f6bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21be737f8f684a87a43e829edcae9df5. Latest consensus state: current_term: 1 leader_uuid: "21be737f8f684a87a43e829edcae9df5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21be737f8f684a87a43e829edcae9df5" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39987 } } peers { permanent_uuid: "207112f995d144c389b884dcb9f6ac83" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 } } peers { permanent_uuid: "8b3137677e8d496ab55d11a56b24f6bd" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } } }
I20260808 23:32:33.191433 10927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b3137677e8d496ab55d11a56b24f6bd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:32:33.193459 10807 log.cc:824] T 00000000000000000000000000000000 P 207112f995d144c389b884dcb9f6ac83: Log is configured to *not* fsync() on all Append() calls
I20260808 23:32:33.194331 10948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "21be737f8f684a87a43e829edcae9df5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21be737f8f684a87a43e829edcae9df5" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39987 } } peers { permanent_uuid: "207112f995d144c389b884dcb9f6ac83" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 } } peers { permanent_uuid: "8b3137677e8d496ab55d11a56b24f6bd" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } } }
I20260808 23:32:33.194685 10948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5 [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:33.196208 10927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b3137677e8d496ab55d11a56b24f6bd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "21be737f8f684a87a43e829edcae9df5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21be737f8f684a87a43e829edcae9df5" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39987 } } peers { permanent_uuid: "207112f995d144c389b884dcb9f6ac83" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 } } peers { permanent_uuid: "8b3137677e8d496ab55d11a56b24f6bd" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } } }
I20260808 23:32:33.196445 10927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b3137677e8d496ab55d11a56b24f6bd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:32:33.197016 10957 catalog_manager.cc:1347] Generated new cluster ID: 317e1a62d8b642eab9609dcb78e14db1
I20260808 23:32:33.197120 10957 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:32:33.214794 10807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 207112f995d144c389b884dcb9f6ac83: No bootstrap required, opened a new log
I20260808 23:32:33.225872 10807 raft_consensus.cc:348] T 00000000000000000000000000000000 P 207112f995d144c389b884dcb9f6ac83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21be737f8f684a87a43e829edcae9df5" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39987 } } peers { permanent_uuid: "207112f995d144c389b884dcb9f6ac83" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 } } peers { permanent_uuid: "8b3137677e8d496ab55d11a56b24f6bd" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } }
I20260808 23:32:33.228299 10957 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:32:33.231469 10957 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:32:33.231706 10807 raft_consensus.cc:729] T 00000000000000000000000000000000 P 207112f995d144c389b884dcb9f6ac83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 207112f995d144c389b884dcb9f6ac83, State: Initialized, Role: FOLLOWER
I20260808 23:32:33.232331 10807 consensus_queue.cc:260] T 00000000000000000000000000000000 P 207112f995d144c389b884dcb9f6ac83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21be737f8f684a87a43e829edcae9df5" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39987 } } peers { permanent_uuid: "207112f995d144c389b884dcb9f6ac83" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 } } peers { permanent_uuid: "8b3137677e8d496ab55d11a56b24f6bd" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } }
I20260808 23:32:33.236690 10985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 207112f995d144c389b884dcb9f6ac83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21be737f8f684a87a43e829edcae9df5" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39987 } } peers { permanent_uuid: "207112f995d144c389b884dcb9f6ac83" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 } } peers { permanent_uuid: "8b3137677e8d496ab55d11a56b24f6bd" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } } }
I20260808 23:32:33.237373 10985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 207112f995d144c389b884dcb9f6ac83 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:32:33.238943 10807 sys_catalog.cc:564] T 00000000000000000000000000000000 P 207112f995d144c389b884dcb9f6ac83 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:32:33.238979 10776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "21be737f8f684a87a43e829edcae9df5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "207112f995d144c389b884dcb9f6ac83"
I20260808 23:32:33.239496 10776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 207112f995d144c389b884dcb9f6ac83 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:32:33.242877 10776 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 207112f995d144c389b884dcb9f6ac83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21be737f8f684a87a43e829edcae9df5 in term 1.
I20260808 23:32:33.245645 10957 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5: Generated new TSK 0
I20260808 23:32:33.245862 10774 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 207112f995d144c389b884dcb9f6ac83 [term 1 FOLLOWER]: Refusing update from remote peer 21be737f8f684a87a43e829edcae9df5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:32:33.246676 10957 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:32:33.247509 10948 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "207112f995d144c389b884dcb9f6ac83" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:32:33.309990 10985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 207112f995d144c389b884dcb9f6ac83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21be737f8f684a87a43e829edcae9df5. Latest consensus state: current_term: 1 leader_uuid: "21be737f8f684a87a43e829edcae9df5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21be737f8f684a87a43e829edcae9df5" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39987 } } peers { permanent_uuid: "207112f995d144c389b884dcb9f6ac83" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 } } peers { permanent_uuid: "8b3137677e8d496ab55d11a56b24f6bd" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } } }
I20260808 23:32:33.309959 11002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 207112f995d144c389b884dcb9f6ac83 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "21be737f8f684a87a43e829edcae9df5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21be737f8f684a87a43e829edcae9df5" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39987 } } peers { permanent_uuid: "207112f995d144c389b884dcb9f6ac83" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 } } peers { permanent_uuid: "8b3137677e8d496ab55d11a56b24f6bd" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } } }
I20260808 23:32:33.310412 10985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 207112f995d144c389b884dcb9f6ac83 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:32:33.310475 11002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 207112f995d144c389b884dcb9f6ac83 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:32:33.314169 11014 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260808 23:32:33.358709 10812 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:32:33.345486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60828 (local address 127.9.44.125:42193)
0808 23:32:33.345662 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:33.345669 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:33.356249 (+ 10580us) server_negotiation.cc:407] Connection header received
0808 23:32:33.357162 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:33.357168 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:33.357261 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:33.357329 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:33.358492 (+ 1163us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60828: BlockingRecv error: recv got EOF from 127.0.0.1:60828 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 23:32:33.358713 10968 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:32:33.345983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36416 (local address 127.9.44.124:32881)
0808 23:32:33.346159 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:33.346165 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:33.356798 (+ 10633us) server_negotiation.cc:407] Connection header received
0808 23:32:33.356995 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:33.357001 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:33.357111 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:33.357222 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:33.358462 (+ 1240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36416: BlockingRecv error: recv got EOF from 127.0.0.1:36416 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 23:32:33.360898 10709 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:32:33.348011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33024 (local address 127.9.44.126:39987)
0808 23:32:33.348203 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:33.348212 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:33.359301 (+ 11089us) server_negotiation.cc:407] Connection header received
0808 23:32:33.359559 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:33.359565 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:33.359648 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:33.359754 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:33.360649 (+ 895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33024: BlockingRecv error: recv got EOF from 127.0.0.1:33024 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 23:32:33.369413 10812 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:32:33.367615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60830 (local address 127.9.44.125:42193)
0808 23:32:33.367783 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:33.367788 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:33.367892 (+ 104us) server_negotiation.cc:407] Connection header received
0808 23:32:33.368140 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:33.368144 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:33.368226 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:33.368306 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:33.369167 (+ 861us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60830: BlockingRecv error: recv got EOF from 127.0.0.1:60830 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260808 23:32:33.381211 10812 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:32:33.373538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60840 (local address 127.9.44.125:42193)
0808 23:32:33.373695 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:33.373700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:33.380123 (+ 6423us) server_negotiation.cc:407] Connection header received
0808 23:32:33.380284 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:33.380287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:33.380344 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:33.380429 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:33.381039 (+ 610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60840: BlockingRecv error: recv got EOF from 127.0.0.1:60840 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 23:32:33.387387 10812 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:32:33.385887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60844 (local address 127.9.44.125:42193)
0808 23:32:33.386029 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:33.386034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:33.386296 (+ 262us) server_negotiation.cc:407] Connection header received
0808 23:32:33.386458 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:33.386462 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:33.386531 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:33.386608 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:33.387223 (+ 615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60844: BlockingRecv error: recv got EOF from 127.0.0.1:60844 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 23:32:33.396124 10812 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:32:33.394355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60850 (local address 127.9.44.125:42193)
0808 23:32:33.394523 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:33.394529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:33.394984 (+ 455us) server_negotiation.cc:407] Connection header received
0808 23:32:33.395149 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:33.395153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:33.395210 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:33.395285 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:33.395937 (+ 652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60850: BlockingRecv error: recv got EOF from 127.0.0.1:60850 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 23:32:33.437552 10812 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:32:33.435293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60862 (local address 127.9.44.125:42193)
0808 23:32:33.435510 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:33.435517 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:33.436231 (+ 714us) server_negotiation.cc:407] Connection header received
0808 23:32:33.436430 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:33.436435 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:33.436504 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:33.436589 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:33.437334 (+ 745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60862: BlockingRecv error: recv got EOF from 127.0.0.1:60862 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260808 23:32:33.458802 10812 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:32:33.456881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60864 (local address 127.9.44.125:42193)
0808 23:32:33.457056 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:33.457062 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:33.457542 (+ 480us) server_negotiation.cc:407] Connection header received
0808 23:32:33.457720 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:33.457724 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:33.457786 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:33.457876 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:33.458585 (+ 709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60864: BlockingRecv error: recv got EOF from 127.0.0.1:60864 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260808 23:32:33.496040 10812 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:32:33.493909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60880 (local address 127.9.44.125:42193)
0808 23:32:33.494094 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:33.494101 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:33.494577 (+ 476us) server_negotiation.cc:407] Connection header received
0808 23:32:33.494772 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:33.494776 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:33.494846 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:33.494940 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:33.495757 (+ 817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60880: BlockingRecv error: recv got EOF from 127.0.0.1:60880 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 23:32:33.565534 10812 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:32:33.563363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60884 (local address 127.9.44.125:42193)
0808 23:32:33.563580 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:33.563587 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:33.564129 (+ 542us) server_negotiation.cc:407] Connection header received
0808 23:32:33.564334 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:33.564339 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:33.564422 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:33.564511 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:33.565310 (+ 799us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60884: BlockingRecv error: recv got EOF from 127.0.0.1:60884 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260808 23:32:33.700672 10812 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:32:33.698511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60894 (local address 127.9.44.125:42193)
0808 23:32:33.698744 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:33.698752 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:33.699136 (+ 384us) server_negotiation.cc:407] Connection header received
0808 23:32:33.699372 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:33.699379 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:33.699467 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:33.699555 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:33.700412 (+ 857us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60894: BlockingRecv error: recv got EOF from 127.0.0.1:60894 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 23:32:33.713454 10948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "21be737f8f684a87a43e829edcae9df5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21be737f8f684a87a43e829edcae9df5" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 39987 } } peers { permanent_uuid: "207112f995d144c389b884dcb9f6ac83" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42193 } } peers { permanent_uuid: "8b3137677e8d496ab55d11a56b24f6bd" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 32881 } } }
I20260808 23:32:33.713783 10948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21be737f8f684a87a43e829edcae9df5 [sys.catalog]: This master's current role is: LEADER
W20260808 23:32:33.962147 10812 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:32:33.960381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60898 (local address 127.9.44.125:42193)
0808 23:32:33.960608 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:33.960614 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:33.960867 (+ 253us) server_negotiation.cc:407] Connection header received
0808 23:32:33.961050 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:33.961054 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:33.961128 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:33.961213 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:33.961936 (+ 723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60898: BlockingRecv error: recv got EOF from 127.0.0.1:60898 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 23:32:34.010264 10941 catalog_manager.cc:1259] Loaded cluster ID: 317e1a62d8b642eab9609dcb78e14db1
I20260808 23:32:34.010365 10941 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8b3137677e8d496ab55d11a56b24f6bd: loading cluster ID for follower catalog manager: success
I20260808 23:32:34.012892 10941 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8b3137677e8d496ab55d11a56b24f6bd: acquiring CA information for follower catalog manager: success
I20260808 23:32:34.014581 10941 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8b3137677e8d496ab55d11a56b24f6bd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 23:32:34.318503 11014 catalog_manager.cc:1259] Loaded cluster ID: 317e1a62d8b642eab9609dcb78e14db1
I20260808 23:32:34.318621 11014 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 207112f995d144c389b884dcb9f6ac83: loading cluster ID for follower catalog manager: success
I20260808 23:32:34.321283 11014 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 207112f995d144c389b884dcb9f6ac83: acquiring CA information for follower catalog manager: success
I20260808 23:32:34.323104 11014 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 207112f995d144c389b884dcb9f6ac83: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 23:32:34.484882 11102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:34.477625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60902 (local address 127.9.44.125:42193)
0808 23:32:34.478264 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:34.478272 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:34.478528 (+ 256us) server_negotiation.cc:407] Connection header received
0808 23:32:34.478703 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:34.478708 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:34.478911 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:34.479053 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:32:34.479484 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:34.480742 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:34.482145 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:34.482863 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:34.482993 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:34.483461 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:32:34.484245 (+ 784us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:32:34.484408 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:34.484459 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 23:32:34.484548 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":282,"threads_started":1}
I20260808 23:32:34.496521 11116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:34.490919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36424 (local address 127.9.44.124:32881)
0808 23:32:34.491543 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:34.491549 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:34.491567 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:32:34.491658 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:34.491687 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:34.491845 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:34.491996 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:32:34.492414 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:34.493113 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:34.494391 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:34.494797 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:34.494907 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:34.495387 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:32:34.495909 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:32:34.496054 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:34.496148 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 23:32:34.496232 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":371,"threads_started":1}
I20260808 23:32:34.521399 11116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:34.503444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36426 (local address 127.9.44.124:32881)
0808 23:32:34.503593 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:34.503599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:34.514026 (+ 10427us) server_negotiation.cc:407] Connection header received
0808 23:32:34.514657 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:34.514688 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:34.514862 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:34.514966 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:32:34.515801 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:34.517024 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:34.519222 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:34.519893 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:34.520011 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:34.520596 (+ 585us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:32:34.520742 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:32:34.520804 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:34.521021 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 23:32:34.521100 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 23:32:34.526583 11102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:34.503295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60910 (local address 127.9.44.125:42193)
0808 23:32:34.508768 (+ 5473us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:34.508774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:34.516308 (+ 7534us) server_negotiation.cc:407] Connection header received
0808 23:32:34.516477 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:34.516482 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:34.516559 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:34.516664 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:32:34.520363 (+ 3699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:34.521726 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:34.524682 (+ 2956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:34.525285 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:34.525415 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:34.525849 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:32:34.525952 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:32:34.526033 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:34.526201 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 23:32:34.526277 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5338}
I20260808 23:32:34.527091 11123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:34.502889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33030 (local address 127.9.44.126:39987)
0808 23:32:34.520305 (+ 17416us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:34.520312 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:34.520330 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:32:34.520434 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:34.520439 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:34.520788 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:34.520955 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:32:34.521708 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:34.522710 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:34.524722 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:34.525122 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:34.525234 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:34.525778 (+ 544us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:32:34.526185 (+ 407us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:32:34.526287 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:34.526724 (+ 437us) server_negotiation.cc:299] Negotiation successful
0808 23:32:34.526813 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17290,"thread_start_us":17587,"threads_started":1}
I20260808 23:32:34.536211 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 10615
I20260808 23:32:34.537237 10699 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:32:34.655921 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 10615
I20260808 23:32:34.683583 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 10715
I20260808 23:32:34.684706 10801 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:32:34.786886 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 10715
I20260808 23:32:34.809062 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 10816
I20260808 23:32:34.810124 10911 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:32:34.914315 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 10816
2026-08-08T23:32:34Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4938 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 23:32:35 dist-test-slave-6bbx krb5kdc[11134](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 23:32:35 dist-test-slave-6bbx krb5kdc[11134](info): set up 2 sockets
Aug 08 23:32:35 dist-test-slave-6bbx krb5kdc[11134](info): commencing operation
krb5kdc: starting...
W20260808 23:32:36.117763 9393 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/krb5kdc/test-user.keytab.
Aug 08 23:32:36 dist-test-slave-6bbx krb5kdc[11134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T23:32:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T23:32:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.44.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
Entry for principal kudu/127.9.44.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab.
I20260808 23:32:36.391134 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:44543
--webserver_interface=127.9.44.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:45379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:44543,127.9.44.125:42563,127.9.44.124:45951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 23:32:36.718125 11151 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 23:32:36.718459 11151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:32:36.718554 11151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:32:36.718662 11151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:32:36.725746 11151 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:32:36.725834 11151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:32:36.725934 11151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:32:36.726001 11151 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:32:36.726083 11151 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:32:36.735642 11151 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:45379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:44543,127.9.44.125:42563,127.9.44.124:45951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.126.keytab
--principal=kudu/127.9.44.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.126:44543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.44.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:32:36.736899 11151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:32:36.738170 11151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:32:36.750766 11161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:36.750829 11158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:36.750811 11159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:32:36.751762 11151 server_base.cc:1034] running on GCE node
I20260808 23:32:36.752538 11151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:32:36.753852 11151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:32:36.755077 11151 hybrid_clock.cc:648] HybridClock initialized: now 1786231956755014 us; error 32 us; skew 500 ppm
I20260808 23:32:36.755393 11151 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:32:36 dist-test-slave-6bbx krb5kdc[11134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231956, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:32:36.758332 11151 init.cc:377] Logged in from keytab as kudu/127.9.44.126@KRBTEST.COM (short username kudu)
I20260808 23:32:36.759697 11151 webserver.cc:460] Webserver started at http://127.9.44.126:40483/ using document root <none> and password file <none>
I20260808 23:32:36.760358 11151 fs_manager.cc:359] Metadata directory not provided
I20260808 23:32:36.760458 11151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:32:36.760753 11151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:32:36.763571 11151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-0/data/instance:
uuid: "fec3a14bf84c4db482ee2a02d47f8abe"
format_stamp: "Formatted at 2026-08-08 23:32:36 on dist-test-slave-6bbx"
server_key: "0655ee7bcca53f19df269e951dbb9b79"
server_key_iv: "690294ae81080e031a0bf3cb84418eaf"
server_key_version: "encryptionkey@0"
I20260808 23:32:36.764398 11151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-0/wal/instance:
uuid: "fec3a14bf84c4db482ee2a02d47f8abe"
format_stamp: "Formatted at 2026-08-08 23:32:36 on dist-test-slave-6bbx"
server_key: "0655ee7bcca53f19df269e951dbb9b79"
server_key_iv: "690294ae81080e031a0bf3cb84418eaf"
server_key_version: "encryptionkey@0"
I20260808 23:32:36.769296 11151 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 23:32:36.772477 11172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:32:36.773651 11151 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260808 23:32:36.773813 11151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-0/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-0/wal
uuid: "fec3a14bf84c4db482ee2a02d47f8abe"
format_stamp: "Formatted at 2026-08-08 23:32:36 on dist-test-slave-6bbx"
server_key: "0655ee7bcca53f19df269e951dbb9b79"
server_key_iv: "690294ae81080e031a0bf3cb84418eaf"
server_key_version: "encryptionkey@0"
I20260808 23:32:36.774015 11151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:32:36.788017 11151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:32:36.791520 11151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:32:36.791800 11151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:32:36.808260 11151 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.126:44543
I20260808 23:32:36.808283 11243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.126:44543 every 8 connection(s)
I20260808 23:32:36.809463 11151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-0/data/info.pb
I20260808 23:32:36.812872 11244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:32:36.815188 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 11151
I20260808 23:32:36.815354 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-0/wal/instance
I20260808 23:32:36.815781 9393 external_mini_cluster.cc:1428] Setting key 2c7fc451e68f1533f50cb4bf3791b153
I20260808 23:32:36.819233 11244 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44543 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:32:36 dist-test-slave-6bbx krb5kdc[11134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231956, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:32:36.841148 11251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:36.822157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46714 (local address 127.9.44.126:44543)
0808 23:32:36.823862 (+ 1705us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:36.823868 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:36.823914 (+ 46us) server_negotiation.cc:407] Connection header received
0808 23:32:36.825652 (+ 1738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:36.825658 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:36.825881 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:36.826029 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:36.827463 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:36.828687 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:36.830455 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:36.830991 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:36.831147 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:36.834502 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:36.834510 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:36.834521 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:36.834551 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:36.837196 (+ 2645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:36.837981 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:36.837987 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:36.837989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:36.838126 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:36.838622 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:36.838625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:36.838627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:36.839083 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:36.839239 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:36.839869 (+ 630us) server_negotiation.cc:299] Negotiation successful
0808 23:32:36.840150 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":1508,"spinlock_wait_cycles":16256,"thread_start_us":1259,"threads_started":1}
I20260808 23:32:36.841207 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:36.822157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:44543 (local address 127.0.0.1:46714)
0808 23:32:36.823025 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:36.823069 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 23:32:36.825296 (+ 2227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:32:36.826129 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:32:36.826136 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:32:36.826809 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:32:36.827243 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:36.827266 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:36.828846 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:36.828851 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:36.830316 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:36.830331 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:36.831228 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:36.831236 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:36.831390 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:36.831992 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:32:36.832015 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:32:36.834284 (+ 2269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:32:36.837337 (+ 3053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:36.837349 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:36.837362 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:36.837871 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:36.838231 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:36.838235 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:36.838238 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:36.838512 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:36.839222 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:32:36.839229 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:32:36.839431 (+ 202us) client_negotiation.cc:769] Sending connection context
0808 23:32:36.839766 (+ 335us) client_negotiation.cc:240] Negotiation successful
0808 23:32:36.840320 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"mutex_wait_us":97,"thread_start_us":373,"threads_started":1}
I20260808 23:32:36.844028 11244 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:32:36.845052 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:36.844558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:42563 (local address 127.0.0.1:50726)
0808 23:32:36.844796 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:36.844929 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 23:32:36.845314 11174 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42563: connect: Connection refused (error 111)
I20260808 23:32:36.846843 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:36.846432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:42563 (local address 127.0.0.1:50734)
0808 23:32:36.846603 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:36.846690 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 23:32:36.847054 11244 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.125:42563: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42563: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.9.44.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.125.keytab.
Entry for principal kudu/127.9.44.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.125.keytab.
I20260808 23:32:36.883124 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.125:42563
--webserver_interface=127.9.44.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.125.keytab
--principal=kudu/127.9.44.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:45379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:44543,127.9.44.125:42563,127.9.44.124:45951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:32:36.896396 11244 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 } attempt: 1
I20260808 23:32:36.897578 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:36.897083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:42563 (local address 127.0.0.1:50746)
0808 23:32:36.897299 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:36.897423 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 23:32:36.898547 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:36.898153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:42563 (local address 127.0.0.1:50758)
0808 23:32:36.898312 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:36.898392 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 23:32:36.898757 11244 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.125:42563: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42563: connect: Connection refused (error 111)
I20260808 23:32:36.967008 11244 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 } attempt: 2
I20260808 23:32:36.968267 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:36.967701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:42563 (local address 127.0.0.1:50766)
0808 23:32:36.967973 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:36.968094 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 23:32:36.969370 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:36.968958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:42563 (local address 127.0.0.1:50774)
0808 23:32:36.969119 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:36.969205 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 23:32:36.969566 11244 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.125:42563: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42563: connect: Connection refused (error 111)
I20260808 23:32:37.060840 11244 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 } attempt: 3
I20260808 23:32:37.062071 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.061519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:42563 (local address 127.0.0.1:50786)
0808 23:32:37.061763 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.061899 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 23:32:37.063143 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.062723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:42563 (local address 127.0.0.1:50788)
0808 23:32:37.062877 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.062968 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 23:32:37.063346 11244 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.125:42563: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42563: connect: Connection refused (error 111)
I20260808 23:32:37.206588 11244 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 } attempt: 4
I20260808 23:32:37.207829 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.207239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:42563 (local address 127.0.0.1:50790)
0808 23:32:37.207531 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.207642 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
W20260808 23:32:37.208281 11255 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 23:32:37.208639 11255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:32:37.208753 11255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:32:37.208873 11255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 23:32:37.208874 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.208462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:42563 (local address 127.0.0.1:50800)
0808 23:32:37.208613 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.208722 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 23:32:37.209052 11244 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.125:42563: Network error: Client connection negotiation failed: client connection to 127.9.44.125:42563: connect: Connection refused (error 111)
W20260808 23:32:37.215916 11255 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:32:37.215989 11255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:32:37.216049 11255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:32:37.216100 11255 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:32:37.216172 11255 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:32:37.225395 11255 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:45379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:44543,127.9.44.125:42563,127.9.44.124:45951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.125.keytab
--principal=kudu/127.9.44.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.125:42563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.44.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:32:37.226501 11255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:32:37.227659 11255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:32:37.238691 11266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:37.238736 11269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:37.238878 11265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:32:37.239485 11255 server_base.cc:1034] running on GCE node
I20260808 23:32:37.240196 11255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:32:37.241401 11255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:32:37.242606 11255 hybrid_clock.cc:648] HybridClock initialized: now 1786231957242535 us; error 42 us; skew 500 ppm
I20260808 23:32:37.242899 11255 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:32:37 dist-test-slave-6bbx krb5kdc[11134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231957, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:32:37.245851 11255 init.cc:377] Logged in from keytab as kudu/127.9.44.125@KRBTEST.COM (short username kudu)
I20260808 23:32:37.247087 11255 webserver.cc:460] Webserver started at http://127.9.44.125:36437/ using document root <none> and password file <none>
I20260808 23:32:37.247726 11255 fs_manager.cc:359] Metadata directory not provided
I20260808 23:32:37.247819 11255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:32:37.248083 11255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:32:37.250870 11255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-1/data/instance:
uuid: "c01c3ec11e514b9dbf81f0841d647e6a"
format_stamp: "Formatted at 2026-08-08 23:32:37 on dist-test-slave-6bbx"
server_key: "cb5826dfe2c2688e0d58e3e008cb04f4"
server_key_iv: "79e0555d05911164892486b14623cdec"
server_key_version: "encryptionkey@0"
I20260808 23:32:37.251565 11255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-1/wal/instance:
uuid: "c01c3ec11e514b9dbf81f0841d647e6a"
format_stamp: "Formatted at 2026-08-08 23:32:37 on dist-test-slave-6bbx"
server_key: "cb5826dfe2c2688e0d58e3e008cb04f4"
server_key_iv: "79e0555d05911164892486b14623cdec"
server_key_version: "encryptionkey@0"
I20260808 23:32:37.256263 11255 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 23:32:37.259346 11277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:32:37.260603 11255 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 23:32:37.260763 11255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-1/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-1/wal
uuid: "c01c3ec11e514b9dbf81f0841d647e6a"
format_stamp: "Formatted at 2026-08-08 23:32:37 on dist-test-slave-6bbx"
server_key: "cb5826dfe2c2688e0d58e3e008cb04f4"
server_key_iv: "79e0555d05911164892486b14623cdec"
server_key_version: "encryptionkey@0"
I20260808 23:32:37.260949 11255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:32:37.272073 11255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:32:37.275444 11255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:32:37.275715 11255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:32:37.292107 11255 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.125:42563
I20260808 23:32:37.292141 11342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.125:42563 every 8 connection(s)
I20260808 23:32:37.293277 11255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-1/data/info.pb
I20260808 23:32:37.296657 11343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:32:37.299690 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 11255
I20260808 23:32:37.299889 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-1/wal/instance
I20260808 23:32:37.300285 9393 external_mini_cluster.cc:1428] Setting key e1720cf5c8e842a42772c9ca22e12ede
I20260808 23:32:37.302815 11343 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44543 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:32:37 dist-test-slave-6bbx krb5kdc[11134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231957, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.125@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:32:37.321516 11251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.305093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46720 (local address 127.9.44.126:44543)
0808 23:32:37.305249 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:37.305253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:37.306455 (+ 1202us) server_negotiation.cc:407] Connection header received
0808 23:32:37.308282 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:37.308288 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:37.308374 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:37.308460 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:37.309860 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.310590 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.312081 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.312493 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.312604 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:37.316083 (+ 3479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:37.316088 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:37.316091 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:37.316123 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:37.318615 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:37.319278 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:37.319282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:37.319285 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:37.319415 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:37.319873 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:37.319876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:37.319878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:37.320148 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:37.320298 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:37.320968 (+ 670us) server_negotiation.cc:299] Negotiation successful
0808 23:32:37.321193 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 23:32:37.322342 11346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.305619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:44543 (local address 127.0.0.1:46720)
0808 23:32:37.306310 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.306345 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 23:32:37.308017 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:32:37.308694 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:32:37.308701 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:32:37.309191 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:32:37.309680 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:37.309698 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.310744 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.310748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:37.311941 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:37.311954 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.312708 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.312716 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:37.312942 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:37.313511 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:32:37.313539 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:32:37.315876 (+ 2337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:32:37.318743 (+ 2867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:37.318752 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:37.318765 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:37.319153 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:37.319522 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:37.319527 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:37.319531 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:37.319758 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:37.320302 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:32:37.320310 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:32:37.320606 (+ 296us) client_negotiation.cc:769] Sending connection context
0808 23:32:37.320841 (+ 235us) client_negotiation.cc:240] Negotiation successful
0808 23:32:37.321288 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":299,"threads_started":1}
I20260808 23:32:37.323999 11343 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.44.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.44.124@KRBTEST.COM" created.
Aug 08 23:32:37 dist-test-slave-6bbx krb5kdc[11134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231957, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.125@KRBTEST.COM for kudu/127.9.44.125@KRBTEST.COM
I20260808 23:32:37.338348 11346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.324588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:42563 (local address 127.0.0.1:50804)
0808 23:32:37.324773 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.324791 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 23:32:37.324888 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:32:37.325882 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:32:37.325887 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:32:37.326270 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:32:37.326392 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:37.326400 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.327690 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.327695 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:37.328512 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:37.328523 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.329262 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.329267 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:37.329427 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:37.329936 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:32:37.329951 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:32:37.333285 (+ 3334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:32:37.336107 (+ 2822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:37.336112 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:37.336116 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:37.336438 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:37.336785 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:37.336790 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:37.336794 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:37.336917 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:37.337462 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:32:37.337467 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:32:37.337608 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 23:32:37.337722 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 23:32:37.337976 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 23:32:37.338476 11350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.324842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50804 (local address 127.9.44.125:42563)
0808 23:32:37.325507 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:37.325512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:37.325536 (+ 24us) server_negotiation.cc:407] Connection header received
0808 23:32:37.325575 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:37.325580 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:37.325732 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:37.325855 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:37.326547 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.327542 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.328643 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.329131 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.329241 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:37.333478 (+ 4237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:37.333489 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:37.333500 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:37.333531 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:37.335996 (+ 2465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:37.336557 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:37.336563 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:37.336565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:37.336673 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:37.337015 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:37.337019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:37.337023 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:37.337338 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:37.337525 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:37.337906 (+ 381us) server_negotiation.cc:299] Negotiation successful
0808 23:32:37.338107 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":373,"threads_started":1}
I20260808 23:32:37.340638 11343 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:32:37.341704 11346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.341292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:45951 (local address 127.0.0.1:59216)
0808 23:32:37.341460 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.341570 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 23:32:37.341992 11280 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
I20260808 23:32:37.343497 11346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.343139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:45951 (local address 127.0.0.1:59224)
0808 23:32:37.343274 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.343375 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 23:32:37.343729 11343 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:45951: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.44.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.124.keytab.
Entry for principal kudu/127.9.44.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.124.keytab.
I20260808 23:32:37.362110 9393 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
/tmp/dist-test-taskzx8VIc/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.124:45951
--webserver_interface=127.9.44.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.124.keytab
--principal=kudu/127.9.44.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.44.84:45379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.44.126:44543,127.9.44.125:42563,127.9.44.124:45951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 23:32:37.393044 11343 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } attempt: 1
I20260808 23:32:37.394227 11346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.393720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:45951 (local address 127.0.0.1:59230)
0808 23:32:37.393927 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.394047 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 23:32:37.395195 11346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.394826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:45951 (local address 127.0.0.1:59236)
0808 23:32:37.394963 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.395063 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 23:32:37.395397 11343 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:45951: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
I20260808 23:32:37.463622 11343 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } attempt: 2
I20260808 23:32:37.464805 11346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.464289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:45951 (local address 127.0.0.1:59246)
0808 23:32:37.464505 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.464614 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 23:32:37.465780 11346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.465405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:45951 (local address 127.0.0.1:59252)
0808 23:32:37.465546 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.465632 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 23:32:37.465989 11343 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:45951: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
I20260808 23:32:37.508291 11244 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 } attempt: 5
Aug 08 23:32:37 dist-test-slave-6bbx krb5kdc[11134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231956, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for kudu/127.9.44.125@KRBTEST.COM
I20260808 23:32:37.523134 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.508907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:42563 (local address 127.0.0.1:50820)
0808 23:32:37.509131 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.509157 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 23:32:37.509312 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:32:37.509717 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:32:37.509722 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:32:37.510275 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:32:37.510547 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:37.510563 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.511524 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.511530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:37.512932 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:37.512950 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.513705 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.513712 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:37.513935 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:37.514544 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:32:37.514566 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:32:37.517196 (+ 2630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:32:37.520908 (+ 3712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:37.520912 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:37.520915 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:37.521284 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:37.521677 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:37.521680 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:37.521683 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:37.521808 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:37.522464 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:32:37.522468 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:32:37.522597 (+ 129us) client_negotiation.cc:769] Sending connection context
0808 23:32:37.522709 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 23:32:37.522896 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 23:32:37.523428 11350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.509046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50820 (local address 127.9.44.125:42563)
0808 23:32:37.509201 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:37.509215 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:37.509234 (+ 19us) server_negotiation.cc:407] Connection header received
0808 23:32:37.509472 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:37.509475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:37.509560 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:37.509636 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:37.510699 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.511386 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.513100 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.513542 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.513633 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:37.517433 (+ 3800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:37.517440 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:37.517445 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:37.517517 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:37.520775 (+ 3258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:37.521412 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:37.521417 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:37.521421 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:37.521580 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:37.521907 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:37.521912 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:37.521915 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:37.522319 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:37.522486 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:37.522856 (+ 370us) server_negotiation.cc:299] Negotiation successful
0808 23:32:37.523090 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 23:32:37.524413 11244 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } has no permanent_uuid. Determining permanent_uuid...
I20260808 23:32:37.525542 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.525184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:45951 (local address 127.0.0.1:59258)
0808 23:32:37.525340 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.525413 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 23:32:37.526490 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.526187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:45951 (local address 127.0.0.1:59266)
0808 23:32:37.526326 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.526401 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 23:32:37.526677 11244 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:45951: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
I20260808 23:32:37.557226 11343 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } attempt: 3
I20260808 23:32:37.558423 11346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.557852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:45951 (local address 127.0.0.1:59272)
0808 23:32:37.558084 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.558210 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 23:32:37.559525 11346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.559023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:45951 (local address 127.0.0.1:59288)
0808 23:32:37.559261 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.559370 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
W20260808 23:32:37.559819 11343 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:45951: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
I20260808 23:32:37.577864 11244 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } attempt: 1
I20260808 23:32:37.578750 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.578335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:45951 (local address 127.0.0.1:59292)
0808 23:32:37.578523 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.578600 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 23:32:37.579648 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.579380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:45951 (local address 127.0.0.1:59308)
0808 23:32:37.579494 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.579555 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 23:32:37.579914 11244 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:45951: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
I20260808 23:32:37.648090 11244 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } attempt: 2
I20260808 23:32:37.649224 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.648771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:45951 (local address 127.0.0.1:59324)
0808 23:32:37.648975 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.649062 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 23:32:37.650096 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.649828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:45951 (local address 127.0.0.1:59334)
0808 23:32:37.649939 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.650001 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 23:32:37.650290 11244 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:45951: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
W20260808 23:32:37.688222 11353 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 23:32:37.688515 11353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 23:32:37.688611 11353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 23:32:37.688712 11353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 23:32:37.695606 11353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 23:32:37.695710 11353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 23:32:37.695806 11353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 23:32:37.695870 11353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 23:32:37.695951 11353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 23:32:37.703063 11343 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } attempt: 4
I20260808 23:32:37.704209 11346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.703657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:45951 (local address 127.0.0.1:43180)
0808 23:32:37.703913 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.704020 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 23:32:37.705200 11346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.704837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:45951 (local address 127.0.0.1:43194)
0808 23:32:37.704988 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.705073 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 23:32:37.705430 11343 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:45951: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
I20260808 23:32:37.706020 11353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.44.84:45379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.44.126:44543,127.9.44.125:42563,127.9.44.124:45951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/krb5kdc/kudu_127.9.44.124.keytab
--principal=kudu/127.9.44.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.44.124:45951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.44.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 22:27:01 UTC on ccycloud.quasar-rdhpdq.root.comops.site
build id 4575491
ASAN enabled
I20260808 23:32:37.707185 11353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 23:32:37.708454 11353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 23:32:37.719602 11365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:37.719643 11363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 23:32:37.719556 11361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 23:32:37.720700 11353 server_base.cc:1034] running on GCE node
I20260808 23:32:37.721417 11353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 23:32:37.722651 11353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 23:32:37.723853 11353 hybrid_clock.cc:648] HybridClock initialized: now 1786231957723783 us; error 41 us; skew 500 ppm
I20260808 23:32:37.724148 11353 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 23:32:37 dist-test-slave-6bbx krb5kdc[11134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231957, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 23:32:37.727041 11353 init.cc:377] Logged in from keytab as kudu/127.9.44.124@KRBTEST.COM (short username kudu)
I20260808 23:32:37.728402 11353 webserver.cc:460] Webserver started at http://127.9.44.124:46787/ using document root <none> and password file <none>
I20260808 23:32:37.729045 11353 fs_manager.cc:359] Metadata directory not provided
I20260808 23:32:37.729147 11353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 23:32:37.729444 11353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 23:32:37.732265 11353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-2/data/instance:
uuid: "18054a09c4f14c2596912f38540e7a62"
format_stamp: "Formatted at 2026-08-08 23:32:37 on dist-test-slave-6bbx"
server_key: "6d4f44ea8f3dfd6294d4b9c4fb116165"
server_key_iv: "1dd05b299be2f97393ae6f7c307349f6"
server_key_version: "encryptionkey@0"
I20260808 23:32:37.733018 11353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-2/wal/instance:
uuid: "18054a09c4f14c2596912f38540e7a62"
format_stamp: "Formatted at 2026-08-08 23:32:37 on dist-test-slave-6bbx"
server_key: "6d4f44ea8f3dfd6294d4b9c4fb116165"
server_key_iv: "1dd05b299be2f97393ae6f7c307349f6"
server_key_version: "encryptionkey@0"
I20260808 23:32:37.737761 11353 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260808 23:32:37.740898 11376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 23:32:37.742147 11353 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 23:32:37.742311 11353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-2/data,/tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-2/wal
uuid: "18054a09c4f14c2596912f38540e7a62"
format_stamp: "Formatted at 2026-08-08 23:32:37 on dist-test-slave-6bbx"
server_key: "6d4f44ea8f3dfd6294d4b9c4fb116165"
server_key_iv: "1dd05b299be2f97393ae6f7c307349f6"
server_key_version: "encryptionkey@0"
I20260808 23:32:37.742506 11353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 23:32:37.756480 11244 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } attempt: 3
I20260808 23:32:37.757499 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.757081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:45951 (local address 127.0.0.1:43196)
0808 23:32:37.757273 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.757354 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 23:32:37.758424 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.758046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:45951 (local address 127.0.0.1:43200)
0808 23:32:37.758248 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.758323 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260808 23:32:37.758625 11244 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.44.124:45951: Network error: Client connection negotiation failed: client connection to 127.9.44.124:45951: connect: Connection refused (error 111)
I20260808 23:32:37.759274 11353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 23:32:37.762758 11353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 23:32:37.762976 11353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 23:32:37.778949 11353 rpc_server.cc:307] RPC server started. Bound to: 127.9.44.124:45951
I20260808 23:32:37.778967 11449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.44.124:45951 every 8 connection(s)
I20260808 23:32:37.780164 11353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-2/data/info.pb
I20260808 23:32:37.783551 11451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 23:32:37.785614 9393 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu as pid 11353
I20260808 23:32:37.785764 9393 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzx8VIc/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786231597224934-9393-0/minicluster-data/master-2/wal/instance
I20260808 23:32:37.786182 9393 external_mini_cluster.cc:1428] Setting key 47656ec0a517d748befe93eed13b4b4f
I20260808 23:32:37.791532 11451 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44543 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:32:37 dist-test-slave-6bbx krb5kdc[11134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.124@KRBTEST.COM
Aug 08 23:32:37 dist-test-slave-6bbx krb5kdc[11134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231957, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.124@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
I20260808 23:32:37.807025 11251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.794154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53374 (local address 127.9.44.126:44543)
0808 23:32:37.794356 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:37.794362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:37.795043 (+ 681us) server_negotiation.cc:407] Connection header received
0808 23:32:37.795229 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:37.795234 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:37.795315 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:37.795417 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:37.796440 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.797087 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.798359 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.798768 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.798875 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:37.802067 (+ 3192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:37.802071 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:37.802074 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:37.802106 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:37.804612 (+ 2506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:37.805253 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:37.805258 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:37.805262 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:37.805397 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:37.805739 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:37.805745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:37.805748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:37.805989 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:37.806126 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:37.806628 (+ 502us) server_negotiation.cc:299] Negotiation successful
0808 23:32:37.806790 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 23:32:37.807432 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.787764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43208 (local address 127.9.44.124:45951)
0808 23:32:37.788551 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:37.788565 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:37.788611 (+ 46us) server_negotiation.cc:407] Connection header received
0808 23:32:37.790851 (+ 2240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:37.790878 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:37.791337 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:37.791786 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:37.792994 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.794614 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.796072 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.796678 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.796849 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:37.800736 (+ 3887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:37.800744 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:37.800754 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:37.800782 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:37.803508 (+ 2726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:37.804179 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:37.804186 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:37.804199 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:37.804336 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:37.804688 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:37.804693 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:37.804696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:37.805161 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:37.805437 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:37.805849 (+ 412us) server_negotiation.cc:299] Negotiation successful
0808 23:32:37.806227 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":291,"threads_started":1}
I20260808 23:32:37.807466 11457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.794144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.126:44543 (local address 127.0.0.1:53374)
0808 23:32:37.794912 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.794949 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 23:32:37.795111 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:32:37.795458 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:32:37.795462 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:32:37.796035 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:32:37.796295 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:37.796304 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.797257 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.797261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:37.798221 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:37.798233 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.798973 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.798980 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:37.799116 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:37.799650 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:32:37.799715 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:32:37.801875 (+ 2160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:32:37.804736 (+ 2861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:37.804747 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:37.804754 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:37.805142 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:37.805499 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:37.805504 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:37.805507 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:37.805625 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:37.806172 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:32:37.806179 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:32:37.806365 (+ 186us) client_negotiation.cc:769] Sending connection context
0808 23:32:37.806554 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 23:32:37.806829 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":411,"threads_started":1}
I20260808 23:32:37.809556 11451 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:32:37 dist-test-slave-6bbx krb5kdc[11134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.126@KRBTEST.COM
Aug 08 23:32:37 dist-test-slave-6bbx krb5kdc[11134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231957, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.124@KRBTEST.COM for kudu/127.9.44.125@KRBTEST.COM
I20260808 23:32:37.823438 11251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.810263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53388 (local address 127.9.44.126:44543)
0808 23:32:37.810426 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:37.810431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:37.810447 (+ 16us) server_negotiation.cc:407] Connection header received
0808 23:32:37.810545 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:37.810549 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:37.810607 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:37.810686 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:37.811730 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.812825 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.813974 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.814633 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.814738 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:37.818582 (+ 3844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:37.818587 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:37.818590 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:37.818626 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:37.821059 (+ 2433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:37.821629 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:37.821634 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:37.821636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:37.821751 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:37.822087 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:37.822092 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:37.822095 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:37.822380 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:37.822629 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:37.822941 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 23:32:37.823131 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 23:32:37.828670 11457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.810095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.125:42563 (local address 127.0.0.1:33914)
0808 23:32:37.810290 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.810306 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 23:32:37.810391 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:32:37.817005 (+ 6614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:32:37.817009 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:32:37.817362 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:32:37.817553 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:37.817565 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.818555 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.818559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:37.819439 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:37.819450 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.820204 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.820209 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:37.820398 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:37.820865 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:32:37.820880 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:32:37.822676 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:32:37.826332 (+ 3656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:37.826337 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:37.826340 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:37.826826 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:37.827213 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:37.827217 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:37.827221 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:37.827399 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:37.827999 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:32:37.828002 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:32:37.828120 (+ 118us) client_negotiation.cc:769] Sending connection context
0808 23:32:37.828224 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 23:32:37.828399 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260808 23:32:37.829011 11350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.816461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33914 (local address 127.9.44.125:42563)
0808 23:32:37.816673 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:37.816679 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:37.816701 (+ 22us) server_negotiation.cc:407] Connection header received
0808 23:32:37.816761 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:37.816766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:37.816842 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:37.816960 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:37.817712 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.818402 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.819581 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.820068 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.820179 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:37.822850 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:37.822855 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:37.822858 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:37.822891 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:37.826195 (+ 3304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:37.826942 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:37.826945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:37.826947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:37.827085 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:37.827513 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:37.827518 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:37.827521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:37.827875 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:37.828033 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:37.828336 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 23:32:37.828561 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 23:32:37.829851 11451 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 23:32:37 dist-test-slave-6bbx krb5kdc[11134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.44.125@KRBTEST.COM
Aug 08 23:32:37 dist-test-slave-6bbx krb5kdc[11134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231957, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.124@KRBTEST.COM for kudu/127.9.44.124@KRBTEST.COM
I20260808 23:32:37.839351 11458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.825121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33926 (local address 127.9.44.125:42563)
0808 23:32:37.825773 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:37.825779 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:37.825804 (+ 25us) server_negotiation.cc:407] Connection header received
0808 23:32:37.825850 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:37.825856 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:37.825924 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:37.826039 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:37.827390 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.828451 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.829807 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.830269 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.830384 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:37.834422 (+ 4038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:37.834428 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:37.834433 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:37.834465 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:37.836866 (+ 2401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:37.837438 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:37.837443 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:37.837447 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:37.837567 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:37.837935 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:37.837940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:37.837944 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:37.838259 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:37.838501 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:37.838763 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 23:32:37.838962 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":292,"threads_started":1}
I20260808 23:32:37.842407 11457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.830488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:45951 (local address 127.0.0.1:43212)
0808 23:32:37.830626 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.830641 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 23:32:37.830831 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:32:37.831087 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:32:37.831092 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:32:37.831446 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:32:37.831565 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:37.831573 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.832470 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.832476 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:37.833603 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:37.833620 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.834369 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.834375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:37.834545 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:37.835010 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:32:37.835024 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:32:37.836774 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:32:37.839969 (+ 3195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:37.839972 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:37.839975 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:37.840378 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:37.840691 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:37.840695 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:37.840698 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:37.840846 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:37.841461 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:32:37.841465 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:32:37.841614 (+ 149us) client_negotiation.cc:769] Sending connection context
0808 23:32:37.841731 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 23:32:37.841991 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 23:32:37.842459 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.830619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43212 (local address 127.9.44.124:45951)
0808 23:32:37.830729 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:37.830732 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:37.830798 (+ 66us) server_negotiation.cc:407] Connection header received
0808 23:32:37.830927 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:37.830931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:37.830987 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:37.831055 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:37.831717 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.832353 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.833739 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.834188 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.834324 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:37.836957 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:37.836962 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:37.836967 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:37.837000 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:37.839844 (+ 2844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:37.840477 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:37.840482 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:37.840486 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:37.840594 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:37.840965 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:37.840969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:37.840972 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:37.841337 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:37.841470 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:37.841825 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 23:32:37.842162 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":102,"server-negotiator.queue_time_us":33}
I20260808 23:32:37.844182 9393 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 23:32:37.848062 11451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18054a09c4f14c2596912f38540e7a62: Bootstrap starting.
I20260808 23:32:37.847165 11207 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "fec3a14bf84c4db482ee2a02d47f8abe"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53388
I20260808 23:32:37.851118 11451 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18054a09c4f14c2596912f38540e7a62: Neither blocks nor log segments found. Creating new log.
I20260808 23:32:37.852162 11451 log.cc:824] T 00000000000000000000000000000000 P 18054a09c4f14c2596912f38540e7a62: Log is configured to *not* fsync() on all Append() calls
I20260808 23:32:37.854964 11451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18054a09c4f14c2596912f38540e7a62: No bootstrap required, opened a new log
I20260808 23:32:37.865125 11451 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18054a09c4f14c2596912f38540e7a62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44543 } } peers { permanent_uuid: "c01c3ec11e514b9dbf81f0841d647e6a" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 } } peers { permanent_uuid: "18054a09c4f14c2596912f38540e7a62" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } }
I20260808 23:32:37.865442 11451 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18054a09c4f14c2596912f38540e7a62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18054a09c4f14c2596912f38540e7a62, State: Initialized, Role: FOLLOWER
I20260808 23:32:37.865868 11451 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18054a09c4f14c2596912f38540e7a62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44543 } } peers { permanent_uuid: "c01c3ec11e514b9dbf81f0841d647e6a" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 } } peers { permanent_uuid: "18054a09c4f14c2596912f38540e7a62" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } }
I20260808 23:32:37.866665 11460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18054a09c4f14c2596912f38540e7a62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44543 } } peers { permanent_uuid: "c01c3ec11e514b9dbf81f0841d647e6a" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 } } peers { permanent_uuid: "18054a09c4f14c2596912f38540e7a62" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } } }
I20260808 23:32:37.866972 11460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18054a09c4f14c2596912f38540e7a62 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:32:37.867861 11451 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18054a09c4f14c2596912f38540e7a62 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 23:32:37.876957 11479 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 18054a09c4f14c2596912f38540e7a62: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 23:32:37.877126 11479 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 23:32:37.874728 11353 master_runner.cc:186] Error getting master registration for 127.9.44.126:44543: OK, instance_id { permanent_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" instance_seqno: 1786231956803538 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 23:32:37.885900 11353 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 23:32:37.935873 11244 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } attempt: 4
Aug 08 23:32:37 dist-test-slave-6bbx krb5kdc[11134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231956, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.126@KRBTEST.COM for kudu/127.9.44.124@KRBTEST.COM
I20260808 23:32:37.953313 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.936657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:45951 (local address 127.0.0.1:43224)
0808 23:32:37.936850 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:37.936871 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 23:32:37.936979 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:32:37.937386 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:32:37.937390 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:32:37.937819 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:32:37.937988 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:37.937998 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.939048 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.939053 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:37.940050 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:37.940063 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.940835 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.940841 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:37.941036 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:37.945208 (+ 4172us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:32:37.945227 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:32:37.947900 (+ 2673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:32:37.950876 (+ 2976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:37.950881 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:37.950886 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:37.951246 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:37.951606 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:37.951611 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:37.951615 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:37.951807 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:37.952365 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:32:37.952370 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:32:37.952525 (+ 155us) client_negotiation.cc:769] Sending connection context
0808 23:32:37.952644 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 23:32:37.952924 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 23:32:37.953349 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:37.936743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43224 (local address 127.9.44.124:45951)
0808 23:32:37.936957 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:37.936963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:37.936988 (+ 25us) server_negotiation.cc:407] Connection header received
0808 23:32:37.937122 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:37.937161 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:37.937229 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:37.937314 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:37.938149 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.938894 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.940200 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:37.940675 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:37.940784 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:37.948071 (+ 7287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:37.948076 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:37.948082 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:37.948113 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:37.950749 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:37.951362 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:37.951366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:37.951368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:37.951501 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:37.951909 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:37.951913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:37.951915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:37.952248 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:37.952410 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:37.952779 (+ 369us) server_negotiation.cc:299] Negotiation successful
0808 23:32:37.952988 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 23:32:37.958050 11244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe: Bootstrap starting.
I20260808 23:32:37.961335 11244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe: Neither blocks nor log segments found. Creating new log.
I20260808 23:32:37.962355 11244 log.cc:824] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe: Log is configured to *not* fsync() on all Append() calls
I20260808 23:32:37.965090 11244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe: No bootstrap required, opened a new log
I20260808 23:32:37.970407 11244 raft_consensus.cc:348] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44543 } } peers { permanent_uuid: "c01c3ec11e514b9dbf81f0841d647e6a" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 } } peers { permanent_uuid: "18054a09c4f14c2596912f38540e7a62" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } }
I20260808 23:32:37.970726 11244 raft_consensus.cc:729] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fec3a14bf84c4db482ee2a02d47f8abe, State: Initialized, Role: FOLLOWER
I20260808 23:32:37.971140 11244 consensus_queue.cc:260] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44543 } } peers { permanent_uuid: "c01c3ec11e514b9dbf81f0841d647e6a" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 } } peers { permanent_uuid: "18054a09c4f14c2596912f38540e7a62" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } }
I20260808 23:32:37.972033 11482 sys_catalog.cc:455] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44543 } } peers { permanent_uuid: "c01c3ec11e514b9dbf81f0841d647e6a" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 } } peers { permanent_uuid: "18054a09c4f14c2596912f38540e7a62" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } } }
I20260808 23:32:37.972393 11482 sys_catalog.cc:458] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:32:37.973023 11244 sys_catalog.cc:564] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:32:37.973181 11207 raft_consensus.cc:482] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 23:32:37.973443 11207 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:32:37.976980 11207 raft_consensus.cc:504] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44543 } } peers { permanent_uuid: "c01c3ec11e514b9dbf81f0841d647e6a" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 } } peers { permanent_uuid: "18054a09c4f14c2596912f38540e7a62" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } }
I20260808 23:32:37.979009 11207 leader_election.cc:288] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe [CANDIDATE]: Term 1 election: Requested vote from peers c01c3ec11e514b9dbf81f0841d647e6a (127.9.44.125:42563), 18054a09c4f14c2596912f38540e7a62 (127.9.44.124:45951)
I20260808 23:32:37.979601 11419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "18054a09c4f14c2596912f38540e7a62"
I20260808 23:32:37.980090 11419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18054a09c4f14c2596912f38540e7a62 [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:32:37.983537 11419 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 18054a09c4f14c2596912f38540e7a62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fec3a14bf84c4db482ee2a02d47f8abe in term 1.
I20260808 23:32:37.984148 11175 leader_election.cc:302] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18054a09c4f14c2596912f38540e7a62, fec3a14bf84c4db482ee2a02d47f8abe; no voters:
I20260808 23:32:37.984612 11482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe [term 1 FOLLOWER]: Leader election won for term 1
I20260808 23:32:37.988241 11482 raft_consensus.cc:686] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe [term 1 LEADER]: Becoming Leader. State: Replica: fec3a14bf84c4db482ee2a02d47f8abe, State: Running, Role: LEADER
I20260808 23:32:37.988799 11482 consensus_queue.cc:237] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44543 } } peers { permanent_uuid: "c01c3ec11e514b9dbf81f0841d647e6a" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 } } peers { permanent_uuid: "18054a09c4f14c2596912f38540e7a62" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } }
I20260808 23:32:37.999711 11497 sys_catalog.cc:455] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe [sys.catalog]: SysCatalogTable state changed. Reason: New leader fec3a14bf84c4db482ee2a02d47f8abe. Latest consensus state: current_term: 1 leader_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44543 } } peers { permanent_uuid: "c01c3ec11e514b9dbf81f0841d647e6a" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 } } peers { permanent_uuid: "18054a09c4f14c2596912f38540e7a62" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } } }
I20260808 23:32:38.000082 11497 sys_catalog.cc:458] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:38.001217 11502 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 23:32:38.004627 11343 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } attempt: 5
I20260808 23:32:38.012109 11504 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260808 23:32:38.014842 11384 tablet.cc:2369] T 00000000000000000000000000000000 P 18054a09c4f14c2596912f38540e7a62: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:32:38.018141 11502 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 23:32:38.019338 11151 master_runner.cc:186] Error getting master registration for 127.9.44.125:42563: OK, instance_id { permanent_uuid: "c01c3ec11e514b9dbf81f0841d647e6a" instance_seqno: 1786231957287483 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 23:32:38.029263 11151 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 08 23:32:38 dist-test-slave-6bbx krb5kdc[11134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786231957, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.44.125@KRBTEST.COM for kudu/127.9.44.124@KRBTEST.COM
W20260808 23:32:38.044202 11180 tablet.cc:2369] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 23:32:38.046833 11346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:38.005168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.44.124:45951 (local address 127.0.0.1:43240)
0808 23:32:38.005429 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 23:32:38.005448 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 23:32:38.005565 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 23:32:38.026174 (+ 20609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 23:32:38.026178 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 23:32:38.026676 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0808 23:32:38.026869 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:38.026882 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.029521 (+ 2639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.029527 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:38.030888 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 23:32:38.030905 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.032517 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.032523 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 23:32:38.032733 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:38.033300 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 23:32:38.033324 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 23:32:38.036480 (+ 3156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 23:32:38.043135 (+ 6655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:38.043142 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:38.043147 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:38.043711 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:38.044466 (+ 755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 23:32:38.044472 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 23:32:38.044476 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 23:32:38.044661 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 23:32:38.045729 (+ 1068us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 23:32:38.045735 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 23:32:38.045897 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 23:32:38.046054 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 23:32:38.046361 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154}
I20260808 23:32:38.048775 11507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:38.005834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43240 (local address 127.9.44.124:45951)
0808 23:32:38.025854 (+ 20020us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:38.025870 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:38.025891 (+ 21us) server_negotiation.cc:407] Connection header received
0808 23:32:38.025933 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:38.025938 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:38.026003 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:38.026099 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:38.027100 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.029306 (+ 2206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.031155 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.032292 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.033072 (+ 780us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:38.036952 (+ 3880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:38.036960 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:38.036965 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:38.037028 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:38.042929 (+ 5901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:38.043923 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:38.043929 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:38.043959 (+ 30us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:38.044264 (+ 305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:38.044860 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:38.044865 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:38.044869 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:38.045514 (+ 645us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:38.047374 (+ 1860us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:38.047610 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 23:32:38.048022 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19874,"thread_start_us":20407,"threads_started":1}
I20260808 23:32:38.054553 11343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c01c3ec11e514b9dbf81f0841d647e6a: Bootstrap starting.
I20260808 23:32:38.056478 11419 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 18054a09c4f14c2596912f38540e7a62 [term 1 FOLLOWER]: Refusing update from remote peer fec3a14bf84c4db482ee2a02d47f8abe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:32:38.056679 11251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:38.001018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53402 (local address 127.9.44.126:44543)
0808 23:32:38.001189 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:38.001195 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:38.010803 (+ 9608us) server_negotiation.cc:407] Connection header received
0808 23:32:38.011812 (+ 1009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:38.011819 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:38.011909 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:38.011985 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:38.018688 (+ 6703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.020025 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.029351 (+ 9326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.030068 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.030175 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:38.040702 (+ 10527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:38.040709 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:38.040714 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:38.040753 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:38.044476 (+ 3723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:38.045368 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:38.045374 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:38.045378 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:38.045563 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:38.048501 (+ 2938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:38.048507 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:38.048511 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:38.049114 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:38.049399 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:38.055982 (+ 6583us) server_negotiation.cc:299] Negotiation successful
0808 23:32:38.056228 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 23:32:38.057541 11497 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe [LEADER]: Connected to new peer: Peer: permanent_uuid: "18054a09c4f14c2596912f38540e7a62" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:32:38.058488 11343 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c01c3ec11e514b9dbf81f0841d647e6a: Neither blocks nor log segments found. Creating new log.
I20260808 23:32:38.059834 11343 log.cc:824] T 00000000000000000000000000000000 P c01c3ec11e514b9dbf81f0841d647e6a: Log is configured to *not* fsync() on all Append() calls
I20260808 23:32:38.062080 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:38.004930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43234 (local address 127.9.44.124:45951)
0808 23:32:38.005102 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:38.005108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:38.015513 (+ 10405us) server_negotiation.cc:407] Connection header received
0808 23:32:38.016584 (+ 1071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:38.016591 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:38.016675 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:38.016743 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:38.024396 (+ 7653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.025542 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.033291 (+ 7749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.034048 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.034145 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:38.051820 (+ 17675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:38.051826 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:38.051831 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:38.051868 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:38.055022 (+ 3154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:38.059130 (+ 4108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:38.059135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:38.059138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:38.059285 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:38.060214 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:38.060219 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:38.060222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:38.060602 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:38.060865 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:38.061496 (+ 631us) server_negotiation.cc:299] Negotiation successful
0808 23:32:38.061726 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:32:38.072433 11458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:38.016108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33936 (local address 127.9.44.125:42563)
0808 23:32:38.016301 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:38.016307 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:38.024065 (+ 7758us) server_negotiation.cc:407] Connection header received
0808 23:32:38.024202 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:38.024207 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:38.024281 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:38.024375 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:38.035041 (+ 10666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.036122 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.045728 (+ 9606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.046456 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.046564 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:38.063715 (+ 17151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 23:32:38.063721 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 23:32:38.063726 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 23:32:38.063765 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 23:32:38.068030 (+ 4265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:38.069408 (+ 1378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:38.069413 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:38.069416 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:38.069590 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 23:32:38.070332 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 23:32:38.070337 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 23:32:38.070340 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 23:32:38.070705 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 23:32:38.070968 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:38.071868 (+ 900us) server_negotiation.cc:299] Negotiation successful
0808 23:32:38.072101 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":64}
I20260808 23:32:38.074491 11343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c01c3ec11e514b9dbf81f0841d647e6a: No bootstrap required, opened a new log
I20260808 23:32:38.080255 11460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18054a09c4f14c2596912f38540e7a62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fec3a14bf84c4db482ee2a02d47f8abe. Latest consensus state: current_term: 1 leader_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44543 } } peers { permanent_uuid: "c01c3ec11e514b9dbf81f0841d647e6a" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 } } peers { permanent_uuid: "18054a09c4f14c2596912f38540e7a62" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } } }
I20260808 23:32:38.080529 11460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18054a09c4f14c2596912f38540e7a62 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:32:38.083235 11497 sys_catalog.cc:455] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44543 } } peers { permanent_uuid: "c01c3ec11e514b9dbf81f0841d647e6a" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 } } peers { permanent_uuid: "18054a09c4f14c2596912f38540e7a62" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } } }
I20260808 23:32:38.083519 11497 sys_catalog.cc:458] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:38.086254 11502 catalog_manager.cc:1347] Generated new cluster ID: e162f8db17f648e182bd317f4a412e61
I20260808 23:32:38.086356 11502 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 23:32:38.091624 11512 mvcc.cc:205] Tried to move back new op lower bound from 7316406100185391104 to 7316406099928596480. Current Snapshot: MvccSnapshot[applied={T|T < 7316406100185391104}]
I20260808 23:32:38.094611 11343 raft_consensus.cc:348] T 00000000000000000000000000000000 P c01c3ec11e514b9dbf81f0841d647e6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44543 } } peers { permanent_uuid: "c01c3ec11e514b9dbf81f0841d647e6a" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 } } peers { permanent_uuid: "18054a09c4f14c2596912f38540e7a62" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } }
I20260808 23:32:38.094998 11343 raft_consensus.cc:729] T 00000000000000000000000000000000 P c01c3ec11e514b9dbf81f0841d647e6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01c3ec11e514b9dbf81f0841d647e6a, State: Initialized, Role: FOLLOWER
I20260808 23:32:38.096905 11460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18054a09c4f14c2596912f38540e7a62 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44543 } } peers { permanent_uuid: "c01c3ec11e514b9dbf81f0841d647e6a" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 } } peers { permanent_uuid: "18054a09c4f14c2596912f38540e7a62" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } } }
I20260808 23:32:38.097138 11460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18054a09c4f14c2596912f38540e7a62 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:32:38.095510 11343 consensus_queue.cc:260] T 00000000000000000000000000000000 P c01c3ec11e514b9dbf81f0841d647e6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44543 } } peers { permanent_uuid: "c01c3ec11e514b9dbf81f0841d647e6a" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 } } peers { permanent_uuid: "18054a09c4f14c2596912f38540e7a62" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } }
I20260808 23:32:38.105270 11517 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01c3ec11e514b9dbf81f0841d647e6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44543 } } peers { permanent_uuid: "c01c3ec11e514b9dbf81f0841d647e6a" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 } } peers { permanent_uuid: "18054a09c4f14c2596912f38540e7a62" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } } }
I20260808 23:32:38.106117 11517 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01c3ec11e514b9dbf81f0841d647e6a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:32:38.107415 11343 sys_catalog.cc:564] T 00000000000000000000000000000000 P c01c3ec11e514b9dbf81f0841d647e6a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 23:32:38.107452 11314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c01c3ec11e514b9dbf81f0841d647e6a"
I20260808 23:32:38.107921 11313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c01c3ec11e514b9dbf81f0841d647e6a [term 0 FOLLOWER]: Advancing to term 1
I20260808 23:32:38.111651 11502 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 23:32:38.113400 11502 catalog_manager.cc:1504] Loading token signing keys...
I20260808 23:32:38.118383 11313 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c01c3ec11e514b9dbf81f0841d647e6a [term 1 FOLLOWER]: Refusing update from remote peer fec3a14bf84c4db482ee2a02d47f8abe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 23:32:38.121598 11314 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c01c3ec11e514b9dbf81f0841d647e6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fec3a14bf84c4db482ee2a02d47f8abe in term 1.
I20260808 23:32:38.124758 11482 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01c3ec11e514b9dbf81f0841d647e6a" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 23:32:38.132432 11502 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe: Generated new TSK 0
I20260808 23:32:38.133322 11502 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 23:32:38.199849 11517 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01c3ec11e514b9dbf81f0841d647e6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fec3a14bf84c4db482ee2a02d47f8abe. Latest consensus state: current_term: 1 leader_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44543 } } peers { permanent_uuid: "c01c3ec11e514b9dbf81f0841d647e6a" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 } } peers { permanent_uuid: "18054a09c4f14c2596912f38540e7a62" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } } }
I20260808 23:32:38.199860 11530 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01c3ec11e514b9dbf81f0841d647e6a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44543 } } peers { permanent_uuid: "c01c3ec11e514b9dbf81f0841d647e6a" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 } } peers { permanent_uuid: "18054a09c4f14c2596912f38540e7a62" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } } }
I20260808 23:32:38.200259 11517 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01c3ec11e514b9dbf81f0841d647e6a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:32:38.200289 11530 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01c3ec11e514b9dbf81f0841d647e6a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 23:32:38.200403 11497 sys_catalog.cc:455] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec3a14bf84c4db482ee2a02d47f8abe" member_type: VOTER last_known_addr { host: "127.9.44.126" port: 44543 } } peers { permanent_uuid: "c01c3ec11e514b9dbf81f0841d647e6a" member_type: VOTER last_known_addr { host: "127.9.44.125" port: 42563 } } peers { permanent_uuid: "18054a09c4f14c2596912f38540e7a62" member_type: VOTER last_known_addr { host: "127.9.44.124" port: 45951 } } }
I20260808 23:32:38.200712 11497 sys_catalog.cc:458] T 00000000000000000000000000000000 P fec3a14bf84c4db482ee2a02d47f8abe [sys.catalog]: This master's current role is: LEADER
I20260808 23:32:38.226590 11542 catalog_manager.cc:1259] Loaded cluster ID: e162f8db17f648e182bd317f4a412e61
I20260808 23:32:38.226765 11542 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c01c3ec11e514b9dbf81f0841d647e6a: loading cluster ID for follower catalog manager: success
I20260808 23:32:38.230495 11542 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c01c3ec11e514b9dbf81f0841d647e6a: acquiring CA information for follower catalog manager: success
I20260808 23:32:38.232975 11542 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c01c3ec11e514b9dbf81f0841d647e6a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 23:32:38.290225 11251 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:32:38.280021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53410 (local address 127.9.44.126:44543)
0808 23:32:38.280219 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:38.280224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:38.288354 (+ 8130us) server_negotiation.cc:407] Connection header received
0808 23:32:38.288573 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:38.288579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:38.288670 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:38.288788 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:38.289960 (+ 1172us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53410: BlockingRecv error: recv got EOF from 127.0.0.1:53410 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 23:32:38.300891 11455 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:32:38.279835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43248 (local address 127.9.44.124:45951)
0808 23:32:38.280037 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:38.280043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:38.290821 (+ 10778us) server_negotiation.cc:407] Connection header received
0808 23:32:38.291080 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:38.291085 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:38.291168 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:38.291240 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:38.300586 (+ 9346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43248: BlockingRecv error: recv got EOF from 127.0.0.1:43248 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260808 23:32:38.310604 11458 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:32:38.308237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33938 (local address 127.9.44.125:42563)
0808 23:32:38.308438 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:38.308444 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:38.309030 (+ 586us) server_negotiation.cc:407] Connection header received
0808 23:32:38.309259 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:38.309265 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:38.309378 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:38.309449 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:38.310346 (+ 897us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33938: BlockingRecv error: recv got EOF from 127.0.0.1:33938 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 23:32:38.362047 11458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:38.344577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33952 (local address 127.9.44.125:42563)
0808 23:32:38.344755 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:38.344761 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:38.350837 (+ 6076us) server_negotiation.cc:407] Connection header received
0808 23:32:38.351322 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:38.351329 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:38.351557 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:38.351735 (+ 178us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:32:38.352455 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.353613 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.358977 (+ 5364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.359699 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.359805 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:38.360595 (+ 790us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:32:38.361230 (+ 635us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:32:38.361380 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:38.361620 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 23:32:38.361709 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260808 23:32:38.406183 11455 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:32:38.396475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43260 (local address 127.9.44.124:45951)
0808 23:32:38.396665 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:38.396672 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:38.404082 (+ 7410us) server_negotiation.cc:407] Connection header received
0808 23:32:38.404630 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:38.404637 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:38.404781 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:38.404865 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:38.405939 (+ 1074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43260: BlockingRecv error: recv got EOF from 127.0.0.1:43260 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 23:32:38.454599 11458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:38.436570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33968 (local address 127.9.44.125:42563)
0808 23:32:38.436760 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:38.436766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:38.447512 (+ 10746us) server_negotiation.cc:407] Connection header received
0808 23:32:38.448039 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:38.448046 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:38.448161 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:38.448231 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:32:38.448942 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.450089 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.452169 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.452889 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.453032 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:38.453686 (+ 654us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:32:38.453837 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:32:38.453903 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:38.454159 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 23:32:38.454244 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 23:32:38.468789 11455 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:32:38.466380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43266 (local address 127.9.44.124:45951)
0808 23:32:38.466551 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:38.466556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:38.467017 (+ 461us) server_negotiation.cc:407] Connection header received
0808 23:32:38.467283 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:38.467289 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:38.467405 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:38.467483 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:38.468527 (+ 1044us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43266: BlockingRecv error: recv got EOF from 127.0.0.1:43266 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 23:32:38.486264 11458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:38.474432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33978 (local address 127.9.44.125:42563)
0808 23:32:38.474592 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:38.474598 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:38.480181 (+ 5583us) server_negotiation.cc:407] Connection header received
0808 23:32:38.480359 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:38.480365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:38.480454 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:38.480558 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:32:38.481021 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.482190 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.484192 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.484865 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.485014 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:38.485487 (+ 473us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:32:38.485622 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:32:38.485731 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:38.485881 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 23:32:38.485968 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 23:32:38.497598 11455 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:32:38.493244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43268 (local address 127.9.44.124:45951)
0808 23:32:38.493419 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:38.493424 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:38.496142 (+ 2718us) server_negotiation.cc:407] Connection header received
0808 23:32:38.496312 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:38.496317 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:38.496410 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:38.496503 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:38.497388 (+ 885us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43268: BlockingRecv error: recv got EOF from 127.0.0.1:43268 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 23:32:38.511265 11458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:38.504872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33994 (local address 127.9.44.125:42563)
0808 23:32:38.505081 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:38.505087 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:38.505105 (+ 18us) server_negotiation.cc:407] Connection header received
0808 23:32:38.505292 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:38.505299 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:38.505386 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:38.505446 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:32:38.506082 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.507264 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.509190 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.509848 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.509979 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:38.510446 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:32:38.510609 (+ 163us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:32:38.510695 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:38.510882 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 23:32:38.510965 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
W20260808 23:32:38.519867 11455 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:32:38.517857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43278 (local address 127.9.44.124:45951)
0808 23:32:38.518123 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:38.518129 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:38.518471 (+ 342us) server_negotiation.cc:407] Connection header received
0808 23:32:38.518621 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:38.518626 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:38.518715 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:38.518814 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:38.519599 (+ 785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43278: BlockingRecv error: recv got EOF from 127.0.0.1:43278 (error 108)
Metrics: {"server-negotiator.queue_time_us":133}
I20260808 23:32:38.540071 11458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:38.533473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34002 (local address 127.9.44.125:42563)
0808 23:32:38.533666 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:38.533672 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:38.534151 (+ 479us) server_negotiation.cc:407] Connection header received
0808 23:32:38.534315 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:38.534320 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:38.534416 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:38.534512 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:32:38.534931 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.536122 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.538013 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.538696 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.538817 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:38.539272 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:32:38.539413 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:32:38.539508 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:38.539657 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 23:32:38.539762 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 23:32:38.555459 11455 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:32:38.553350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43290 (local address 127.9.44.124:45951)
0808 23:32:38.553506 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:38.553512 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:38.554025 (+ 513us) server_negotiation.cc:407] Connection header received
0808 23:32:38.554180 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:38.554185 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:38.554279 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:38.554396 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:38.555252 (+ 856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43290: BlockingRecv error: recv got EOF from 127.0.0.1:43290 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 23:32:38.580317 11458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:38.574307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34016 (local address 127.9.44.125:42563)
0808 23:32:38.574501 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:38.574507 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:38.574801 (+ 294us) server_negotiation.cc:407] Connection header received
0808 23:32:38.574937 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:38.574943 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:38.575036 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:38.575144 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:32:38.575500 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.576760 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.578329 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.578989 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.579116 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:38.579542 (+ 426us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:32:38.579726 (+ 184us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:32:38.579841 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:38.579964 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 23:32:38.580048 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260808 23:32:38.588881 11455 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:32:38.587008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43292 (local address 127.9.44.124:45951)
0808 23:32:38.587193 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:38.587198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:38.587501 (+ 303us) server_negotiation.cc:407] Connection header received
0808 23:32:38.587701 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:38.587706 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:38.587780 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:38.587857 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:38.588676 (+ 819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43292: BlockingRecv error: recv got EOF from 127.0.0.1:43292 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 23:32:38.634554 11458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:38.625102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34018 (local address 127.9.44.125:42563)
0808 23:32:38.625295 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:38.625302 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:38.625770 (+ 468us) server_negotiation.cc:407] Connection header received
0808 23:32:38.626265 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:38.626271 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:38.626338 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:38.626426 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:32:38.626924 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.627778 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.632379 (+ 4601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.632938 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.633043 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:38.633635 (+ 592us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:32:38.633842 (+ 207us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:32:38.633951 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:38.634103 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 23:32:38.634183 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260808 23:32:38.642071 11455 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:32:38.639977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43306 (local address 127.9.44.124:45951)
0808 23:32:38.640173 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:38.640179 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:38.640750 (+ 571us) server_negotiation.cc:407] Connection header received
0808 23:32:38.640908 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:38.640912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:38.640980 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:38.641056 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:38.641867 (+ 811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43306: BlockingRecv error: recv got EOF from 127.0.0.1:43306 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 23:32:38.714546 11458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:38.709648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34034 (local address 127.9.44.125:42563)
0808 23:32:38.709829 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:38.709836 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:38.710115 (+ 279us) server_negotiation.cc:407] Connection header received
0808 23:32:38.710307 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:38.710311 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:38.710379 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:38.710459 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:32:38.710898 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.711586 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.712940 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.713345 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.713458 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:38.713843 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:32:38.713972 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:32:38.714048 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:38.714196 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 23:32:38.714277 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 23:32:38.721619 11455 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:32:38.719889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43312 (local address 127.9.44.124:45951)
0808 23:32:38.720101 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:38.720106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:38.720366 (+ 260us) server_negotiation.cc:407] Connection header received
0808 23:32:38.720526 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:38.720530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:38.720598 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:38.720680 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:38.721447 (+ 767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43312: BlockingRecv error: recv got EOF from 127.0.0.1:43312 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 23:32:38.861614 11458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:38.856925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34042 (local address 127.9.44.125:42563)
0808 23:32:38.857111 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:38.857115 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:38.857560 (+ 445us) server_negotiation.cc:407] Connection header received
0808 23:32:38.857739 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:38.857742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:38.857815 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:38.857893 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:32:38.858219 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.858905 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.860265 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:38.860682 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:38.860769 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:38.861083 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:32:38.861181 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:32:38.861241 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:38.861348 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 23:32:38.861412 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 23:32:38.868853 11455 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 23:32:38.866992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43326 (local address 127.9.44.124:45951)
0808 23:32:38.867200 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:38.867205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:38.867500 (+ 295us) server_negotiation.cc:407] Connection header received
0808 23:32:38.867692 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:38.867698 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:38.867780 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:38.867864 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 23:32:38.868639 (+ 775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43326: BlockingRecv error: recv got EOF from 127.0.0.1:43326 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 23:32:38.879159 11479 catalog_manager.cc:1259] Loaded cluster ID: e162f8db17f648e182bd317f4a412e61
I20260808 23:32:38.879235 11479 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 18054a09c4f14c2596912f38540e7a62: loading cluster ID for follower catalog manager: success
I20260808 23:32:38.881722 11479 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 18054a09c4f14c2596912f38540e7a62: acquiring CA information for follower catalog manager: success
I20260808 23:32:38.883443 11479 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 18054a09c4f14c2596912f38540e7a62: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 23:32:39.136191 11458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:39.131343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34058 (local address 127.9.44.125:42563)
0808 23:32:39.131511 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:39.131516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:39.131822 (+ 306us) server_negotiation.cc:407] Connection header received
0808 23:32:39.131988 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:39.131991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:39.132068 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:39.132152 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:32:39.132555 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:39.133249 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:39.134578 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:39.134982 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:39.135101 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:39.135488 (+ 387us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:32:39.135593 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:32:39.135684 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:39.135836 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 23:32:39.135916 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 23:32:39.146378 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:39.141303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43332 (local address 127.9.44.124:45951)
0808 23:32:39.141518 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:39.141524 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:39.141661 (+ 137us) server_negotiation.cc:407] Connection header received
0808 23:32:39.141797 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:39.141800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:39.141962 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:39.142056 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:32:39.142442 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:39.143153 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:39.144426 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:39.144827 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:39.144917 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:39.145307 (+ 390us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:32:39.145857 (+ 550us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:32:39.145967 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:39.146037 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 23:32:39.146115 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 23:32:39.158275 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:39.151559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43340 (local address 127.9.44.124:45951)
0808 23:32:39.151735 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:39.151741 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:39.152664 (+ 923us) server_negotiation.cc:407] Connection header received
0808 23:32:39.152918 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:39.152923 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:39.152999 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:39.153059 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:32:39.153934 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:39.154815 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:39.156584 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:39.156977 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:39.157081 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:39.157518 (+ 437us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:32:39.157649 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:32:39.157740 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:39.157943 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 23:32:39.158026 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 23:32:39.159307 11458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:39.152669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34068 (local address 127.9.44.125:42563)
0808 23:32:39.152838 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:39.152843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:39.153434 (+ 591us) server_negotiation.cc:407] Connection header received
0808 23:32:39.153604 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:39.153608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:39.153685 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:39.153761 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:32:39.154439 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:39.155357 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:39.157402 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:39.157857 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:39.157965 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:39.158526 (+ 561us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:32:39.158652 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:32:39.158711 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:39.158965 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 23:32:39.159041 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 23:32:39.159586 11711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 23:32:39.151519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53416 (local address 127.9.44.126:44543)
0808 23:32:39.152094 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0808 23:32:39.152100 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 23:32:39.152454 (+ 354us) server_negotiation.cc:407] Connection header received
0808 23:32:39.152712 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 23:32:39.152719 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 23:32:39.152940 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 23:32:39.153108 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 23:32:39.153820 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:39.154615 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:39.156629 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 23:32:39.157262 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 23:32:39.157385 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 23:32:39.158001 (+ 616us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 23:32:39.158442 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 23:32:39.158544 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 23:32:39.159204 (+ 660us) server_negotiation.cc:299] Negotiation successful
0808 23:32:39.159288 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":320,"threads_started":1}
I20260808 23:32:39.163915 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 11151
I20260808 23:32:39.165000 11237 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:32:39.273402 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 11151
I20260808 23:32:39.302661 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 11255
I20260808 23:32:39.303776 11338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:32:39.408612 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 11255
I20260808 23:32:39.436498 9393 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu pid 11353
I20260808 23:32:39.437628 11445 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 23:32:39.543397 9393 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzx8VIc/build/asan/bin/kudu with pid 11353
2026-08-08T23:32:39Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4626 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9565 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (362219 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST